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Abstract. We study a continuous time Bayesian quickest detection problem in which observation 
times are a scarce resource. The agent, limited to making a finite number of discrete observations, 
must adaptively decide his observation strategy to minimize detection delay and the probability of 
false alarm. Under two different models of observation rights, we establish the existence of optimal 
strategies, and formulate an algorithmic approach to the problem via jump operators. We describe 
algorithms for these problems, and illustrate them with some numerical results. As the number of 
observation rights tends to infinity, we also show convergence to the classical continuous observation 
problem of Shiryaev. 

1. Introduction 

Problems in quickest detection, also known as online change point analysis and disorder detection, 
have been studied for much of the twentieth century. In such problems, one observes a channel of 
information whose statistical properties abruptly change at some unknown instant in time. The ba- 
sic problem is to determine when this change occurs, using only the observations from the channel. 
The mathematical theory of quickest detection begins with [21], who studied a Bayesian formula- 
tion of the problem in discrete time, in which the disorder time possesses some prior probability 
distribution. To describe this problem more precisely, we suppose that on some probability space 
we have a random variable Q taking nonnegative integer values, modeling the disorder time, and 
a sequence of random variables Zi,Z2,.-- modelling the observations. Conditionally on the set 
{Q = i}, the random variables Zi, Z2, ■ ■ ■ , -^i-i have the distribution Pi, and the random variables 
Zi, Zi+i, . . . have the distribution P2, where Pi and P2 are distinct but equivalent probabilities. The 
random variable is supposed to have a geometric distribution. With probability vr, = 0, and 
P(0 = n) = (l — -7r)(l — p)^~^p, for some constant p between zero and one. Let F be the filtration 
generated by Zi, Z2, ■ ■ ■■ For a F-stopping time r, the risk 

P^{t) = P^{t <e) + cE [(r - G)+] . 

Here, P'^(r < 6) represents the probability of false alarm, E [(t — Q)~^] is the expected delay 
time, and c is a constant which weighs the relative importance of these two terms. The goal in the 
quickest detection problem is to find a stopping time r which minimizes the risk. 

The first formulation of a quickest detection problem in continuous time is also found in |21j . 
Here, one observes a process X with dXt = dBt + al^t>Qjdt, where Bt is a standard Brownian 
motion, and a is constant. G is a random variable which is zero with probability vr, and with 
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probability 1 — vr it is exponentially distributed with parameter A. Both Bt and are not directly 
observable. As in the discrete-time formulation, the goal is to estimate the value of as a result 
of observing X. Our objective is to minimize a weighted average of the probability of false alarm 
and the detection delay time. The set over which we optimize is the set of stopping times for the 
filtration generated by X. 

Since their original formulation, there has been an extensive literature on quickest detection 
problems which modify or generalize the classical assumptions of [21j. We mention a few of these 
papers, although it is impossible to describe the entire literature on the subject. In |12|, the 
deterministic drift term a received after disorder is replaced with a random variable. In [13], the 



In 



"linear" penalty for delay, E [(r — Q)"*"] , is replaced with an exponential one, E 
|15] . the continuous time problem is solved on a finite time horizon, in comparison with the classical 
infinite horizon case. In continuous time, it is also natural to study a Poisson process whose intensity 
rate suddenly changes, and this problem, known as Poisson disorder, has been extensively studied 
in, for example, [T9]. [TO]. [9]. and [8]. 

In all of the above problems, there is perfect observation. Another interesting generalization of the 
quickest detection problem occurrs when we place restrictions on our ability to make observations. 
The literature on this problem is relatively sparser, but there is a strand of research from the 
1960's and 1970's in which observations are available at all times, but must be purchased. In [2], 
the costly information quickest detection problem for Brownian Motions is first studied; solutions 
are not obtained, but some qualitative properties of the value function are established. In [6], the 
author studies a problem with both costly information and imperfect observations; again, no explicit 
solutions are found. Finally, in [3], a more thorough analysis of a costly information problem is 
attempted, but there is a problem with the author's analysis: when the process X is not continuously 
observed, the posterior process (e.g. vrt = P (0 < t\J^^)) ceases to become a sufficient statistic, 
and the time elapsed since an observation was last made must also be tracked. More recently, in 
[1], a minimax problem is studied in discrete time, where the expected detection delay is minimized 
subject to constraints on false alarm probability and expected number of observations taken before 
the disorder time, and in [5] the same problem is studied in a Bayesian framework. 

Alternatively, one may formulate a quickest detection problem in which observations of the 
process X can be made only at discrete time periods, and such that the agent possesses, no matter 
what, a fixed amount of observation rights. For example, we could imagine a remote battery 
powered sensor which has enough power to make a fixed number of observations and must be 
active for an extended amount of time. Such a problem was first studied in |14j . in which it is 
assumed that observations fall on a grid which is determined exogenously. If observations can 
be made only at discrete time periods, it makes sense to consider the case when there is control 
over when observations can be used: if observations are a limited resource, then the judicious use 
of them should increase efficiency significantly. In such a scenario, the observation times will no 
longer be exogenously given, but will be determined adaptively within the problem as part of the 
optimal strategy. In |14j, an infinite sequence of of observation times is given. If we allow the 
controller to choose when observations are made, it does not make sense to allow him infinitely 
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many observation rights, because as we will see, such a problem is degenerate, and equivalent to 
the classical continuously observed case. Therefore, if observation times can be chosen, there must 
be some limit on how they can be spent. In this paper, building off the discrete-time analysis in 
we study such a problem: an agent sees to determine when a disorder occurs, but his ability 
to observe is constrained. In the first variant of the problem, we assume that the agent receives a 
lump sum of n observation rights which he may use as he sees fit. In the second variant, we assume 
that an independent Poisson process regulates the times at which new observation rights become 
available. 

We now outline the structure of the paper. In Section[2j we formulate the lump sum n-observation 
problem, and establish the theoretical existence of optimal strategies. In Section [3j we demonstrate 
that as n — 7- oo, this n-observation problem converges to the classical continuous observation prob- 
lem. In Section [4j we formulate the stochastic arrival rate n-observation problem, and establish 
the theoretical existence of optimal strategies. In Sections [5] and [6j we give a numerical algorithm 
for computing the value functions and optimal strategies in the lump sum n-observation problem, 
and illustrate some results from the implementation of this algorithm. In Section [TJ we describe a 
heuristic algorithm for computing the value functions and optimal strategies in the stochastic ar- 
rival rate problem, and illustrate a result from a partial implementation of this algorithm. Sections 
[8j[9j and 10 contain the technical proofs of the results in Sections [2| [4j and[3j Finally, Appendix [A] 



establishes the dynamics of the posterior process under discrete observations. 
2. The Lump Sum n-OesERVATiON Problem: Setup, Existence of Optimal Strategies 

Our basic setup is a probability space (0, J^, P'), which supports a Wiener process X = {Xt}t>o 
and an independent random variable 0, which has the same distribution as before: with probability 
p it is zero, and with probability 1 — p it is exponentially distributed with parameter A. 

In [ID], observation times are determined exogenously, and therefore the information flow is a 
fixed aspect of the problem. In contrast, when the agent must decide when to make observations, 
the information flow is itself variable. In other words, the filtration is dependent on the observation 
strategy used. We therefore have to be somewhat technical in our definition of observation strategies. 
We will now inductively define elements in the set of allowed observation strategies, denoted by D". 

Definition 2.1. We say that a sequence of random variables ^ = {ipi,ip2, ■ ■ ■ iV'n} G i?" if i^i ^ 
^2 < • • • < -i/^n, ^1 deterministic, and for 1 < j < n, Vj £ 

m (t(X^j, . . . , X^._-^,iIji, . . . jipj-i), i.e. ipj is measurable with respect to the sigma algebra generated 
by X^^, . . . . . . ,ipj-i. We set ipQ = 0, and for convenience take tpn+i = oo. 

For each G D", let T^, = a{X^-^ , ■ ■ ■ , X^. ,ipi, . . . generates a continuous time filtration 

= [J^j^)t>o in the following way. We say that A G J-j^ if and only if for each 1 < j < n, 
{tpj < t} G J'^.- Intuitively, this just means that the set A is known at time t if, for any j, it is 
known at the time of the j^^ observation when this observation comes before t. Let be the set 
of F*-stopping times which are a.s. finite. 
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Let <I>"^ be the conditional odds-ratio ratio process that the disorder has occurred, supposing that 
the observation strategy $ has been used. In other words 

^ ^ P(9 < t\Ff) 

* p{e>t\Ff)' 

The posterior process can be calculated recursively by the following formula, starting from 
^0 ~ T^- fo'^ more details, please see Appendix [a| which follows the derivation on p. 32-33 of (TO]. 

where AV'n = i>n- V'n-i, = X^^ - X^^_^, ip{t, <j)) = e'^*(0 + !)-!, and 

(2.2) j(At,</.,z) 



2 \ ^ />Af /' / \ 2 2 



(2.3) =exp<j azVAt+ ( A- — ) At J>(/>+ / A exp <j ( A + j - J> du. 







According to Lemma 3.1 of |10j, the minimum Bayes risk equals Rn{p) = 1— p)cV^(p/(l- 
p)), where 



(2.4) VnU) = inf inf 

and the expectation is with respect to a probability measure P under which X is a standard 
Weiner process and = (p. Consequently, (2.4) is the problem we will focus on. 

Proposition 2.2. Let ^ G D", and let r he an -stopping time. Then for each < j < n, 
T'^{ipj<T<tpj+i} ^'^^ {^i — < V'i+i} ^'^G both J^^^ , -measurable. 

Proof. The proof is done by a basic modification of Proposition 3.1 and Theorem 3.2 of [14J. The 
essential property here is that between observations, there is no flow of new information. □ 

Define 

= {r G T* : for (xi G $7 with t{u!) < ■i/'n,(w), r(w) = ipj{oo) for some < j < n| , 

i.e. those F*-stopping times that do not stop between observations. The following proposition says 
that, in contrast with [1^, it is never optimal to stop between observations: if one has a total of n 
observations at their disposal, he may as well use all of them. 

Proposition 2.3. K(0) = _ini^ iiif^ E'^ [/J^ e"^* ($f - f ) dt] . 

Proof. See Section [8| □ 

Note that for each <I>* evolves deterministically between observations. This means that 
between observations, there is no additional information being accrued. Therefore, upon making 
an observation, one may as well determine in that instant when to make the next observation, as 
opposed to waiting to see what happens e seconds in the future; no additional information is gained 
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by waiting. Therefore, the problem is amenable to study by the recursive use of jump operators. 
We lay out this strategy now. 

For bounded w : M+ — )■ M, define the operators 

(2.5) Kw{t,^) ^ r uj{j{t,^,z)f-^^t^dz, 



2n 

(2.6) Jw{t, ^) ^ e-^" (^ifiu, <P) -^^du + l^,^Qye~^'Kw{t, 0), 
and 

(2.7) Jow{())) = miJw{t,())). 

Set vo{(j)) = JqO. Inductively define the value functions, for j > 1, 

Vj{cl)) = JoUj-l(0). 

Let < /c < n, and let = {V'f , . . . , V^} G O''. We set 

(2.8) D"(^I/'=)^{vI/ = {V'i,...,V'n}GO":Vi = V'^', l<i<kY 

These are the observation strategies whose first k observation times agree with those in We 
define the following conditional value functions: 



7^(\I/'^) = essinf essinf E 



Propositions 2.4 and 2.5 allow us to describe the optimization problem in terms of functions 
defined by the jump operator Jq. It also establishes that the optimization problem is Markov. 



Proposition 2.4. For any n, < k < n, and ^''^ G , 

Proof. See Section [8j □ 
Proposition 2.5. For any n, 

7fc (^') < Vn-k {^t) ■ 
Hence, 7^ (*'') = fn-fc {^Xk) ' ^"^ particular, Vn{4') = Vn{4>)- 

Proof. See Section [8j □ 
For < A; < n and e > 0, define 

K~ki<P) - min{s > : Jvn-k{s, 4>) < JQVn-k{4>) + e}- 

These functions are used to construct the near optimal strategies needed for the proof of Proposition 
but we must show that they are measurable. Note that in the definition of h^_f^, we require 



2.5 



the first time s such that Jvn-k{s, 4>) ^ Jo'Vn-k{4') + £• If we simply required any such e-optimal 
time, we could use a Measurable Selection Theorem, as in [23], to imply the measurability of hf^-k- 
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Such a theorem, however, would provide only abstract existence. For computational reasons it is 
preferable to take the first optimal time. 

Lemma 2.6. For < k < n and e > 0, ~ V'fc + ^n-fc (^**^) ^ stopping time, i.e. it is 

measurable with respect to . 

Proof. See Section [8| □ 

Corollary 2.7. Fix n > 1 and e > 0. Consider the observation strategy '^'^ = '^''{(j)) — {^f , . . . , V'nl; 
defined inductively by ^\ = /i.^(</>), and for 2 < j < n, = + I I • Let t'^{oj) = 

inf{V^j(a;) : = ^j+i(a;),0 < j < n-1} A (^^(u;) + t^ (^|:(^))) G where tl{4>) is defined 

to satisfy ip{tQ{(l)) , (p) = ^. Then 



ne. 



lo V 

3. Convergence to the Continuous Observation Problem 

In this section, we will show the extended weak convergence of a discretized quickest detection 
problem to the (classical) continuous observation quickest detection problem, as formulated in [20], 
Chapter 4. In all of these problems, the cost functional has the same form, while the dynamics 
of the underlying odds processes capture the effect of different observation procedures. The the- 
ory of extended weak convergence, as developed by Aldous in [Ij, provides a metric under which 
convergence of optimal stopping problems and their value functions are guaranteed. 

To be more precise, we will show that in a sequence of discrete-time problems, the odds processes 
extended weak converge to the continuous observation odds process <I>^. We consider two discrete 
time problems which are essentially equivalent, one of which fits the model of [14j. Studying these 
problems will give upper bounds for the value function of our (adaptive) n-observation problem 
because they are more restrictive with respect to admissible observation and stopping strategies: 
in our n-observation problem, there is complete freedom over both observation and stopping times, 
whereas in |14j there is freedom over the stopping time but observations are confined to a preset grid. 
As we will see, the value function fc(</>) in the continuous observation problem always gives a lower 
bound for our value functions Vn{4>)- Therefore, we can construct a sequence of functions {v^}n>i 
such that v^ic/)) > Vni(p) > ^^c('A) '^{4') ~^ Vc{(j)), which suffices to show that Vni4') i Vc{4>). 

3.1. Review of the Continuous Observation Problem and Comparison to the Lump 
Sum n-Observation Problem. As before. Let X be a standard Brownian motion which gains 
drift a at the unobservable time O, satisfying P(0 = 0) = p, P(6 G dt\Q > 0) = e~'^*. Let F'^ be 
the filtration generated by X, and S'^ the set of associated stopping times. In the quickest detection 
problem with continuous observation, the minimization problem is 

R^{p)= inf P{t <Q) + cE[{T-e)+]. 



QUICKEST DETECTION WITH DISCRETELY CONTROLLED OBSERVATIONS 



7 



For details on this problem, see |20j, Chapter 4. Here, P and E refer to a probability measure 
under which P(0 = Q) = p. As in Proposition 2.1 of [lOj, we may write 

i?^(p) = 1 _ p + (1 _ p)cVc 

where Vc ( ^3- ) = Vc{4>) = inf E [rf° (<I>^ — ^) dt \ , and <I>^ is the odds process under continuous 
observation. The dynamics of $f are given by the following stochastic differential equation, whose 
derivation is in |20j : 

(3.1) d^l = \{l + ^l)dt + a^ldWu 

with W a standard Brownian Motion. The quickest detection problem is therefore reformulated 
as an optimal stopping problem on the diffusion The following proposition is intuitively clear, 
since continuous observation is certainly preferable to being limited to a finite set of observation 
times. Recall the value function f «((/>) of Section [2| 

Proposition 3.1. For each n, Vn{(l)) > Vc{4>)- 

Proof. Let n > be fixed. Let "if = {ipi, . . . , tpn} be an admissible observation strategy, as described 
in Section [2] ^' induces the filtration F*, along with its set of stopping times 5*. As in |14j . the 
optimal stopping problem associated with the observation strategy \I' is 

where = P(r < 9) + cE[{t - 9)+]. By definition, {J^^)^yo = F'^' C = (J7)t>o, in the 

sense that J^* C J-f for each time t. It follows then that 5* C S'^. Therefore, R^{p) > R'^{p). 
Writing R^{p) = 1 - p + (1 - p)cuvi/ (t3^), it follows that (^j^^ > Vc (t^^)- Let D"- denote 
the set of all admissible n-observation strategies. Then 

Vn{(l)) = inf V^{(j)) > Vc{4>). 

□ 

3.2. Defining tlie Discretized Problem, and tlie Convergence Result. We will define two 
closely related processes, and Let At = ^. The process will be defined on the grid 

points {0, At, 2At, . . .}, and then it will be extended to M+ as a piecewise constant function. Let 
{Zi, Z2, . . .} be a sequence of i.i.d A^(0, 1) random variables. We define and ^'^^^ recursively, so 
that they only differ in between grid points. 

Definition 3.2. Define the process 

.5r = $i'fc_i)A* 




for G N, 

for {k - l)At < t < kAt. 
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Definition 3.3. Define the process (J)®'"; 

^^kAt = j{^t^z,,^Ti)^t) for ken, 

$f = ^ (A(t -{k- l)At), $5:1)^,) for (k - l)At < t < kAt, 

where (^X{t - {k - l)At), ^^^'^^^^^^^ = e^(*-('=-i)^*) (S^^'^^j^^ + l) - 1. We remark that the dy- 
namics of <I)'^'" are precisely those of our n-observation problem when observations are taken every 
^ units of time. Since the gaps between observations are deterministic, they are also the typical ex- 
ample of the model in |14j . The dynamics of are modified to make computations more tractable. 
Notice also that and induce the same filtration. We take F*^ to be the (continuous time) 
natural filtration generated by <I>", and the set of F^-stoping times. We set 



l^icl>) 4 inf E 



To properly state our result, we need the concept of extended weak convergence, from [IJ. We 
state the definition for the sake of completeness, but we will essentially only need the fact that 
extended weak convergence implies convergence of optimal stopping problems. 

Definition 3.4. Let {X,¥) he a random process, considered as a random element in D{M.-^.), the 
set of cadldg paths on M+. For each t, there exists a conditional distribution Zt for X , given Tt, 
and Zt may be viewed as a random element of V{D{M-^-)), the set of probabilities on D(M+). It 
is a fact (see Theorem 13.1 of that these Zt can be combined to form a cddlag process taking 
values in V{D{M.^)) . This process Z is referred to as the prediction process. For processes (^"', F") 
and (X,¥), we say that X" extended converges to X, writing X"^ ^ X, if the associated prediction 
processes converge weakly to Z, i.e. weak convergence of their induced measures on 'P(D(M+)). 

Our principal interest in extended weak convergence is derived from the following (in a slightly 
weakened form) theorem in [T|. Let 7 : [0,oo) x M — t- M be bounded and continuous. Given a 
process (X, F), let 71 denote its stopping times bounded in size by L, and define 

r(L)^ supi?[7(T,XT)]. 

Tan 

Proposition 3.5. [Theorem 17.2, (Aldous)J Suppose (X",F") ^ (X°°,F°°). Suppose {X°°,¥°°) 
is quasi left continuous (or continuous), and suppose that¥°° is the usual filtration for X°° . Then 

r„(L) ^roo(L). 

Our goal, therefore, is to show that <I>" ^ The following two results from [1] yield a feasible 
strategy for establishing extended weak convergence to a diffusion. The effectiveness of this method 
lies in the fact that the (complicated) limiting process never needs to be directly studied; this is 
the basic property of establishing weak convergence. The message of these two Propositions is the 
following: the standard way that one shows weak convergence is Proposition |3.6[ but in fact weak 
convergence is strictly weaker than the conditions in this Proposition. On the other hand, it turns 
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out that these conditions are exactly equivalent to extended weak convergence. Thus, by following 
the "standard" method for establishing weak convergence, one obtains the more powerful extended 
weak convergence for free. 

Proposition 3.6 (Theorem 8.22, (Aldous)). Let a{x) > andb{x) be bounded continuous functions 
and let xq G M. Let X be the diffusion with drift b{x) and variance a{x), and Xq = xq. Let (X", F") 
be a sequence of processes. Suppose that for all L > 
(a) XS_^Xo 

v2 



(b) E 



sup (Xf 

t<L 



x: 



t-j 



as n 



oo. 



Suppose also that for each n, there exist and A/"" adapted to F" such that for all L > 

(c) (M", F") is a martingale, where = - b{X^)ds - iVj" 

(d) (5",F") is a martingale, where = {M^f - Jla{X^)ds-Mp 

(e) sup E [(iV^) 



TeT" 



-7-0 as n —7- oo 
(f) sup E [lAA^I] as n oo, 
where T£ is the set of Y"^- stopping times bounded by L. Then X'^' 



X (i.e., weak convergence). 



Proposition 3.7 (Proposition 21.17, (Aldous)). Let (y",F") be a sequence of processes, and X the 
diffusion with drift b{x) and variance a{x). In order that Y"^ ^ X (i.e. extended weak convergence) , 
it is necessary and sufficient that there exist X^ adapted to F" such that 



(i) sup |Xr - Y," 



in probability 



t<L 



(ii) (X^jF") satisfies the hypotheses of Proposition 3.6 



Proposition 3.8. As n 



oo, <l>f 



Proof. The proof consists of checking the six conditions in Proposition 3.6 which necessitates 

□ 



establishing some moment inequalities on <I>'^. We refer the reader to Section^ 



Corollary 3.9. As n 

Proof Let 5'„(0) = C" 
Note that 



oo, ^>®'" 
> + l)-l 



sup |<I> 

0<t<L 



gn 



max 



As in the proof of Proposition 3.8 is a submartingale, and so by Doob's L Inequality, 



E 



sup I <1>" 

0<t<L 



t I 



o 



1 



E 



1 + 



Using the moment bounds on established in Section [9| we see that this last quantity above 
is O (^). Now, we can see that Condition (i) in Proposition 3.7 is satisfied. Applying it with 
Proposition 3.8 we deduce the Corollary. □ 

Corollary 3.10. As n ^ oo, v^{4>) — ^ vdcj)) and Vn{(p) — ^ Vc{4>). 
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Proof. First, note that for any e > 0, there exists a L = L(e) such that for ah n, 

^{(t>) > mf E \ r e-^' f $f - -) ds 
TeT",T<L Uo \ ^) . 

and the same type of inequahty holds true for Vc{4>)- This is because the running reward function 

at time s is greater than — ^e"'*''^, as and are nonnegative. Therefore, the value functions 

and Vc are uniformly approximated by problems where the allowed stopping times are uniformly 

bounded. Therefore, to show that converges to Vc, we may assume that all stopping times are 

bounded by some constant L. 



Now, we cannot apply Proposition 3.5 directly, since the value functions and Vc are optimal 
stopping problems, not on and ^>'^, but on their time integrals. Fortunately, there is a simple 
way to work around this technical difficulty, using one last result from [Ij. 

Lemma 3.11. Let H : L'(M) — )• L'(M) he a continuous mapping such that if f{u) = g[u) for u < t 
then iHf){t) = {Hg){t). Then if (X",F") ^ (X°^,F°°) and = H{X''), (y",F") ^ (y°°,F'^ 



For H defined by {Hf){t) = Jq f{s)ds, it is clear that the conditions of Lemma 



3.11 



are satis- 



fied, at the very least when H is resticted to continuous paths. Therefore, '"'ds ^ Jq ^'^ds. 



Therefore, by Proposition 3.5 we have — >• Vc- In computing v^, we take [Ln\ observations, so 
Vn > v\^Ln\ ^ '^c- By the monotonicity of Vn with respect to n, it follows that — >• Vc- □ 

Remark 3.12. Once convergence to the continuous observation case has been established, the nat- 



ural question to ask next concerns the convergence rate. As indicated by Table 6.1, this convergence 
rate is quite slow. Using the results in [16], we would expect that, as n —>• oo, 

— Vr\ = O ( n~2 



The results of [16] apply to diffusions which are uniformly elliptical. The diffusion defined in 



(3.1) has vanishing volatility at zero, hut its behavior at the stopping threshold is elliptic, and this 
fact, combined with its upward drift, should allow one to extend the results of [I6] to Also, the 
numerical scheme introducted in [7], as a combination of Monte Carlo and a finite difference scheme 
for solving obstacle problems, is quite relevant. The convergence rate proof here could be adapted 
to give the rate of convergence of the value functions in our setup. The problem here would be to 
obtain similar results with the non- degeneracy assumption. We note that these techniques allow for 
establishing the same convergence rate of value functions in Dayanik's model and our own adaptive 
observation model. It is likely the case that these two methods have the same convergence rate, but 
with different constants. A more sophisticated analysis will be required to uncover these differences. 
In this direction, the theory of adaptive discretization of SDE's may be useful (see |17j . |18j }. since 
we may interpret an observation strategy as a kind of discretization of the diffusion 

Additionally, it is relevant to study the differences between deterministic observation and adap- 
tive observations without letting n — oo. The efficiency difference between these two observation 
strategies is most crucial precisely when there is a relatively small amount of observation rights. 



For numerical results in this direction, see Figure 6.4 
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4. The Stochastic Arrival Rate ti-Observation Problem: Setup, Existence of 

Optimal Strategies 

We will consider two subcases of this problem. First, we assume that a total of n observation 
rights arrive via a Poisson process. Second, we assume that the rates arrive indefinitely from a 
Poisson process. The second case will be addressed as a limiting case of the former. Suppose 
that, in addition to supporting a Wiener process X and the random variable 6, the space {^l,P) 
supports an independent, completely observable Poisson process {Nt}t>o with arrival rate > 0. 
Let T]i < rj2 < • • • denote the increasing sequence of jumps times of A'^. For convenience, take 
??o = 0. We will define the set of allowed observation strategies for both the "n total observation 
rights" problem and for the infinite observation rights problem. When we consider the "n total 
observation rights" problem, we will stop N after n arrivals, and assume that rjn+i = oo. As before, 
we will first define the set of admissible observation strategies. 

Definition 4.1. For a sequence of random variables V'l ^ V'2 ^ " " " ^ V'n; we say ^ = {ipi, ■ ■ ■ , ipn} 
is an admissible observation strategy in the stochastic arrival rate n-observation problem, written 
^ £ if 

ipj e m a{X^^,. . . ,X^^_-^,ipi, . . . ,V'j-i,r/i, . . .,7]j) and ipj > r]j 
for each 1 < j < n. For convenience, we will always set = for any ^. 

Definition 4.2. For ^ = {^|Jl,■^p2, ■ ■ ■}, we say that ^ is an admissible observation strategy in the 
stochastic arrival rate infinite observation problem, written ^ G 0°° , if for each n, {ipi, . . . ,ipn} G 

Using the same construction as in the previous section, each ^ € O", 1 < n < oo, induces a 
continuous time filtration F* = {J-^)t>o which is built up from the discrete observations made at 
times V'i- We take F* = F* V F^, F^ being the filtration generated by the Poisson process N. 
Each induces the set of F*-stopping times and the observed posterior process defined by 



(l2Tj). 

Take 1 < n < oo. As before, according to Lemma 3.1 of [lOj . the minimum Bayes risk equals 
Rn(p) = l- p+ (1 - p)cVnip/{l - p)), where 

(4.1) V„(0) ^ inf inf Ef \ [ e"^* - ^ 

and the expectation E'l'[-] is with respect to a probability measure P under which A is a standard 
Weiner process and <I>q = cf). Hence, we will focus on solving (4.1). 



We will first specialize to the case where n < oo. As in the previous section, in considering the 
problem V„, we can optimize over a smaller set of stopping times than T*. 

Definition 4.3. Let ^' G £)", and let t e T'^ . We say that t e if {ipi < t < V'i+i} n {^i > 
rji+i} = 0, for each < i < n — 1. 
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Note that {ijji > ryj+i} represents the scenarios when, after making observation i, the agent has 
additional observation rights stockpiled. Therefore, a stopping time r G 7^* is one that does not 
stop while there are unused observation rights. As in Section [2], we have 

Proposition 4.4. V„{(p) = Jni^ inl^ E't' [£ e"-^* {^f - ^) dt] . 

Proof. Let ^ G £)", and r G T*. First, note that every stopping time r G T* satisfies {^o < t < 
^/^i} n {V'o ^ '?i} = 0) simply because {^o ^ '?i} = 0) "^i being a strictly positive random variable. 
The proof now is essentially identical to that of Proposition 2.2 As before, if we were to stop the 
game while having unused observation rights, we could construct a new observation strategy which 
adds in an additional observation at that stopping time, without changing the value received. □ 

Now, we will define some operators, which have analogs in the lump sum n-observation problem. 
Let A^ C M5_ denote the set of feasible values of the state process {t, <I>t). Precisely: 



A^ 



{{y,<P):y>0,<P>e^y-l} 



Here y represents the time since the last observation. In the absence of observations, the trajectory 
of the state process follows the path [t, e^^{(j) + 1) — 1), starting from (j) at time zero. Since (p >Q 
at time zero, all trajectories must lie in A^. 

Recall the operator K from (2.5). We will extend it as follows: for w : — )• M bounded, define 

(4.2) Kw{t,^) ^ r nj{0,j{t,4>,z)f-^^^t^dz. 



/2vr 

In the next two operators, the "0" superscript stands for "no observations stockpiled". We define, 
for w : — )■ M bounded, 

J°w{t,y,(l)) 

(4.3) ^ 1^ /ie-^" (^^"''* e-^^ (^ip{r, <P) " ^) + e'^n^.^^ywiy + u, ip{u, ^))^ du, 

(4.4) 4w{y,(l))^mfj'^w{t,y,ct)). 

Let us explain the operator j". It describes the situation in which the agent has no observations 
stockpiled, the posterior is (p, and y units of time have passed since the last observation was made. 
Faced with this scenario, he stops at time t, which may be prior to the arrival time u of the next 
observation right, or after it. An agent will be left with no observations stockpiled only if he has 
just used an observation, so for these operators y will effectively be zero. For subsequent operators, 
we will consider scenarios where y is positive, and so for this reason we keep the notation consistent. 

Next we define jump operators J"*" and , corresponding to the scenario when the agent has 
stockpiled observation rights after he has either just made an observation or received an observation 
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right. We define, for w^,w'^ : — t- M bounded, 

(4.5) J+iw\w^){t,y,. 

puAt ^ 

(p{r, (p) 

+e-^h{t^.^yKw^ [y + t, ifi-y, 0)) + e-^"l|„<t|u;2(y + u, ip{u, </.)) j du, 

(4.6) 4{w\w^){y,(l))^mf J+{w\w^){t,y,(t>). 




From Proposition 4.4 we have seen that it is never optimal for an agent to stop while he has unused 
observation rights. Therefore, if he has observation rights stockpiled, the agent either observes 
immediately (t = 0), which is equivalent to stopping, or chooses his next observation time t > 0. If 
u is the next arrival time of an additional observation right, then his next observation time t may 
be either prior to or after the arrival of the next observation right. Here of the two continuation 
functions and nP' , corresponds to this former scenario, and w'^ to the latter. The variable y 
denotes the amount of time that has passed since the agent has last made an observation, which 
may be nonzero if an observation right has arrived more recently than the last time an observation 
was made. 

We will need one more pair of operators, corresponding to the times when all n observation rights 
have been received. Note that this scenario explains why the "lump sum n observation rights" 
problem is essentially embedded in this one. Therefore, note the similarity between J*^, Jq, defined 



below, and J, Jq, defined in (2.7). The main difference consists in allowing y to be nonzero, allowing 
for the possibility that time has elapsed since the last observation. We define, for w : — )• M 
bounded, 

(4.7) rw{t, y, <t>) = 1^ e~^''- (^(^(r, ^) --)jdr + e-^'Kw{y + t, ^{-y, c^)), 

(4.8) JSw{y,^)^Mrw{t,y,(l)). 

Fix 1 < n < oo. Set <^„+i(y,(/.) ^0. For < A; < n, set <,fc(y,0) = J^vl^j,^^{y,(P). The 
superscript "n" corresponds to n total observation rights, while the subscript "n,k" corresponds 
to n observation rights received and k observations used. Note that when there are n observation 
rights arriving stochastically, it is the case that once all of these n observations have arrived, we 
essentially revert to the lump sum problem. We now define „_;^(y, 0) = Jgt'JJ „_i(y, </>) and, 
for < /c < n — 1, i^{y, (j)) = Jq{v^^i v^kliu, </>)• Proceeding inductively in this way, we 
define 

vl^{y, (t>) ^ Jo°(^^+ij)(y, </>), < j < n, 
vlk{y,<i)) ^ J+{vlk+^,v^+^^f,){yA),^< k<j<n. 
The function v'^^{y,(j)) is a value function, representing the value when there are n total obser- 
vation rights, of which j have been received and k < j spent, the current posterior level is (f), and 
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y units of time have elapsed since the last observation was made. Note that by definition of Jg, 
v^j ^ 0, ^ j ^ n. From this and the definition of Jq , it also follows that t)"^ < for all j and k. 
We illustrate the relationship between the value functions and the jump operators through figures 
O and 1121 



Figure 4.1. Schematic of Stochastic Arrival Problem, n = 4 




Fix < k < n, and let = {V'l , • . . , V'fc} ^ from Definition 4.1 We set, for k < j < n, 
Olki^^) ^ = {Vi, . . . , V'n} G : = V'f, 1 < i < ^ and Vfc+i > Vj} ■ 

Intuitively, 0^j^{"^^) consists of the observation strategies one can pursue after observing at . . . , ip^, 
and refraining from observing next until tjj. Note that the last requirement V'fc+i ^ Vj is vacuous 
when j = k,k + 1. We let, for < k < n and k < j < n, 



k\ A 



ess inf E 



ess inf 



C I ' '^fc^'^J 



Note that the "reference" time above is ^\)\ V r\j. We are in a scenario where j observation rights 
have been received and k spent; if > rjj, we arrived at this state from "j observation rights 
received, k — l observations spent", and if rjj > ip^, we arrived at this state from "j — 1 observations 
received, k observations spent". 



Proposition 4.5. For any n, 0<k<j<n and = {il^i, . . . , tp^} E , 
(4.9) > vl, {^Pl V - ^Pt ^J^v,^ 

on the set {tp^ < r/j+i}. 
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Figure 4.2. Recursive Computation of Value Functions, n = 4 



start 



V, 



0,0 





r+ 










r+ 











r+ 












r+ 













r+ 
















Proof. See Section 10 



□ 



For the proof of the other inequahty, we wih need to construct some optimal stopping times, 
describing when one should either observe the process or stop and accept the change hypothesis. We 
will do this inductively, with the help of some auxiliary functions. Set s" ^{y, (/)) = sJJ = to (</>), 
defined by 



(4,10) ,;(^)=i,„,(_^)vo. 

For < A; < n, define o'^ f^{y,(p) = inf |s > : J'^t'JJ ^^.^^^(s, y, 0) = '/o''^JJ^fc+i(y' We define, for 
< i < n, 

s]j{y,<p) ^ inf {s > : J%J^,^^{s,y,(t>) = Jo%;+ij(y, 0)} 
and, for < j < n, < A; < j, 

olkiy,^) = inf {s > : J+ (s, y, </>) = Kfc+i'^i+i,fc) (2^''^)} • 

The notation "s" and "o" stands for, respectively, stop, and observe. This is in line with the 
reasoning that one should stop only when there are no available observation rights, i.e. j = k. 
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For ^''^ = , . . . , il^^} G O^, we define the "action times" (either stopping or making 
observation) t^/^, k < j < n. Set 

^ Vfc V r/„ + oik (V'^ V r?„ - 
and we will inductively define, on the set {^^ < k < j < n, 



'j + l,k 



and 



if V ry, + oj^, (V.,^ V 77, - <i>g^^^ 



> 



Proposition 4.6. For any n,0<k<j<n and = {^Ji, ■ ■ ■ , V'fc} G , on the set {ip^ < r/j+i 



(4.11) 



Hence, 7"fc(^'^) = i'"^ [jp^ ~ V'fc; ^gvr? j ' Furthermore, on the set {t/jj < rjj+i}. 



(4.12) 0,cl>, 



g-A(^-Vj) 



and /or k < j, on the set {ip^ < r/j+i}, 

(4.13) ^;-:.(^,'v,7,-^^$Sv„) 



V»7i 



/n particular, Vn{(j>) = o('/')- 



Proof. See Section 10 
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As a consequence of Proposition 4.6, we may inductively describe the optimal observation strate- 
gies and stopping times, which are as follows. Consider a given instant of time, when an observation 
has just been spent or an observation right has just been received. Let j be the number of obser- 
vation rights received, k < j he the number of observation rights used, let (p be the current value 
of the posterior process, and let y be the amount of time elapsed since the last time an observation 
was made. 

Corollary 4.7. The following observation/stopping strategy is optimal. Suppose that an observation 
has just been made or an observation right has just been received. 
Observation Strategy 

(1) If k = j , there are no available observation rights. Wait until time rjj^i, increment j by 1, 
and proceed to (2) with the appropriate changes to (p,y. 

(2) If k < j , calculate t^^, which is a function of j, k, (p, y, and the current time. IfT^]^ < Vj+i) 
spend an observation right at time 7^^, and increment k by 1. If k + 1 < j, proceed to (2) 
and if k + 1 = j, proceed to (1), making the appropriate changes to (p and y. Otherwise, if 
T^k — Vj+i) increment j by 1, and proceed to (2) with the appropriate changes to (p and y. 

Stopping Strategy Ifk=j andrj- < r/j+i, stop the game at time t^-. IfT^j > flj+ij increment 
j by 1 and proceed to Observation Strategy (2). The game is never stopped when k < j. 

Remark 4.8. In the corollary above, we say that the agent optimally stops the game only when 
he has no spare observation rights. This is essentially a formalism. One can envision a scenario 
in which the agent makes an observation, and notices that the posterior is at a very high level, 
indicating that it is very likely that the disorder has occurred. The agent will want to stop the 
game immediately. In our setup, the agent, if he has spare observation rights, will exercise them 
all instantaneously to get to the point where he has no observation rights remaining, after which he 
will stop the game. 

4.1. The Infinite Horizon Problem. We consider now the subcase of the stochastic arrival 
problem in which observation rights continue to arrive indefinitely. The following proposition says 
that the value function in the infinite arrival problem is approximated uniformly by the value 
function in the n arrival problem, as n goes to infinity. Since the strategy space for the n arrival 
problem is contained within the strategy space for the infinite arrival problem, it therefore follows 
that we may use optimal strategies in the n arrival problem to find near optimal strategies in the 
infinite arrival problem. 

Proposition 4.9. The value functions Vra(</>) converge to as n ^ oo, uniformly over <p. 

More precisely, < Foo - K < M 77T) 



Proof. The inequality Kxd < is an immediate consequence of the fact that is naturally 
included in , i.e. for any element ^ of there is an element ^ of such that the first n 
observation times of ^ coincide with those of ^. 

For the second inequality, let ^ = {ipi,ip2^ . . •} be an arbitrary element of £)°°. By definition, it 
must be the case that ipn+i > rjn+\, and that {ipi, . . . , ipn} is an element of Noting that for all 
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the posterior process <I>* is positive, it follows that 

1 



(4.14) V^-Vn>E 



oo \ 



-E 
c 



Now, r]n+i, being the sum of n + 1 independent exponential random variables with parameter has 

/ X n+1 

the Erlang distribution ij-n+i ~ Erlang(n + 1,/i), which has Laplace transform /*(s) = ij^^ 

n+1 



It therefore follows that the right hand side of (4.14) above is equal to ^ ijj^j , which tends to 



zero as n — )• oo. □ 

Remark 4.10. A similar argument may be used to show the convergence o/ q(0, 0) ( n total 
observations arriving stochastically, of which none have yet arrived) to (n total observations, 

all of which are available), as the arrival rate — )• oo. Suppose that for some (j), t^((/)), the optimal 
time to make the first observation in the lump sum n-observation problem, is strictly positive. Using 
the cumulative distribution of an Erlang random variable, it is easily calculated that the probability 
that all n observation rights arrive before time t^(</>) is 

71—1 ^ 

k=0 

When n and (j) are fixed, so that t^{(j)) is fixed, this expression converges to 1 almost exponentially 
fast as ^ ^ oo. If all observation rights arrive before time t*^{(j)), then the stochastic arrival of 
the observation rights imposes no restriction on observation strategies vis-a-vis the scenario in 
which all n-observation rights are available all along, because the agent has received all observation 
rights by the time he wishes to make even a single observation. Therefore, with probability at least 
1 - ^^Zo Me"^*"^'^^(/^C(</'))*' the strategy that one would pursue in the Lump Sum n-observation 
problem is also feasible in the stochastic arrival rate problem. This implies that for fixed n and 
(j), VQQ(0,(f)) should converge at least almost exponentially fast to Vn{(t>) as ^ ^ oo. Note that a 
uniform rate of convergence over all (j) is not guaranteed. When t^{(j)) is very close to zero, it 
becomes increasingly important to have observation rights immediately available. Additionally, this 
argument does not hold uniformly over all n as n ^ oo. In fact, the convergence rate o/?;^q(O,0) 
to Vc{(j)) as a function of fi will be comparable to the convergence of Vn{4>) to Vc{(p) as a function 



of n, which is rather slow (see Table 6.1). This is because, in any finite time interval, the expected 



number of received observation rights will be proportional to the arrival rate fi. 

5. An Algorithm for the Lump Sum n-OBSERVATiON Problem 

In this section, we explicitly describe an algorithm for computing the value functions vo,vi, . . . ,vn, 
as well as the boundaries which determine when observations should be made. We give a rigorous 
construction which shows how solutions may be constructed up to any specified error tolerance. 
Actual code, in which certain heuristics are used to speed up implementation, will be given in ??. 
We have the following main result in this section, giving worst case error bounds: 

Proposition 5.1. Fix a positive integer N. Then in O(^) function evaluations, we may uniformly 
approximate vo{(p),vi{4>), . . . ,fAr(0) to within e. 
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We note that in the process of calculating the value functions, we also determine the boundaries 
which determine the optimal observation behavior. We outline the steps of the algorithm in Sub- 
section [5]T1 In Subsection|5.2[ we justify the error bounds of Step 2 of the algorithm. In Subsection 



5.3, we explain the error bounds of Step 3, as well as explaining how an upperbound (j) may be 
constructed. Finally, in Subsection |5.4[ we give error bounds for iterating Steps 2 and 3 multiple 
times. 

5.1. Pseudo-code for the Algorithm. Here we outline the steps of the algorithm. Subsequent 
parts of this section will explain why such an algorithm works to uniformly approximate the value 
functions. 

(1) Fix N , the total number of observations. Discretize the (j) variable into 0o = 0, 01, . . . , 0j = 
0, and set all value functions equal to zero for (p > (p. 

(2) The function vo{(f)) can be analytically computed. Fix (pj, and approximately minimize 
t I— JvQ{t, (pj) by computing Jw{ti, (pj), iov Iq = 0,ti, . . . ,tK = T a discretization of t, and 



T an upper bound on the size of optimal t, established in Lemma 5.4 Let the minimizer 



be t*„{^j). 

(3) Having computed above an approximation to vi{(pj), interpolate these values in a piecewise 
constant fahion to obtain a function vi{(p) which approximates fi (</>)• 

(4) Let the collection of points {t^{(pj),ip{t'^{(pj),(pj)) define the observation barrier. 

(5) Repeat Steps 2, 3, except now minimizing 1 1— )■ Jvi{t,(p), to obtain an approximation V2{(p) 

to V2{(p). 

(6) Continue this procedure until V]\f{(p) is computed. 
5.2. Minimizing 1 1— )• Jw{t,<p) for (p Fixed. 
Lemma 5.2. For each n> 0, — < Vn{(p) < for all (p. 



Proof. According to Proposition 2.5 Vn = Vn, where V-i = 0, and for n > 0, Vn = Jofn-i- Here Jq 
is the jump operator defined in Section [2] Note that clearly satisfies the conclusion of the lemma. 
Therefore, it suffices to establish the inductive step: if < ti„ < 0, then — ^ < JoVn ^ 0. The 
upper bound follows from JoVn{(p) < Jvn{0, (p) = 0. For the lower bound, calculate that for any t, 

JVn{t,(p) 

e~^" {^{u, (p) -^du + e'^^Kvn{t, cp) 







c / \ c 



_ -1 

c 

Taking the infimum over all t yields J(}Vn{(p) > — • D 
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Lemma 5.3. For each n > 0, Vn{(j)) is concave and increasing in (j). 

Proof. Follows by definition, and the fact that an infimum of concave functions is again concave. □ 
Lemma 5.4. For T = ^{l + ^) + ^ and each n, 

Vn{(t)) = JoVn-l{4>) 



inf 

0<i<T 



ip{u,(j)) ) (in + e ^*Kvn-i{t,< 



In other words, the optimal time t* can be assumed to be less than or equal to T . 
Proof. Note that 



-Am 



ip{u,4>) 



A 



du 



+ 1 



-Am 



1 + 



A 



du 



1 



A 



It follows therefore, that for t > ^ (l + ^) + ^ = T, Jvn{t,4>) > for any n. Here we have used 
the uniform lower bound for Vn established in Lemma |5.2[ By the upper bound in that Lemma, 
^ 0, so it is sufficient to minimize Jvn{t,(/)) over t E [0,T]. □ 

Lemma 5.5. Let \\ ■ \\Lip denote the Lipschitz norm. For each n > 0, ||fn||Lip <T + \ \vn-i\\Lip- 



Proof. Take (pi < (j)2. We have, using Lemma 5.4 for the first inequality, 

ft 



\Vn{(t> 



Vn{(l>2)\ < sup 
0<t<T 







sup 

0<t<T 



-Am 



e^^* {KVn^l{t,(l)l) - KVn-lit,(l)2)) 



(p{u,(f)i) - - 



-Am 



(p{u,(f)2) 



du 



We treat these two terms on the right hand side separately. We calculate that the first term is 
actually equal to 



{4>i - (t>2)du 

To calculate the second term, fix t G [0,T]. Then 



sup 

0<t<T 



-xt 



(vn-l{j{t,(l)i,z)) - Vn-l{jit,(p2,z))^ ^ dz 



— I I'l'n— 1 1 1 Lip 6 



-Ai 



\j{t,(j)i,z) - j{t,<j)2,z)\- 



2tt 

2^ 



■dz 



I ^n— 1 1 iLip^ 



-Af I 



-Ail 



-z^ 12 

^azVt+(\-a'^/2)t^ 



27r 



dz 



\Vn-l\\Lipe \(pi - (p2\e 
\Vn-l\\Lip\(t>l - (t>2\-, 
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where the first equahty uses the definition of j{t, (j), z), in Section [2] It now follows that \vn{4>i) 

Vn{4'2)\ <{T+ ||t;„_i||Lip) - (p2\- 

Lemma 5.6. The mapping t 1— t- Jvn{t,<p) is ^-Holder continuous. In particular, \^Jvn{t, (f>)\ < 
4> + a + bWvnWLipt^^^"^ , for constants 



□ 



1 + - + - I e 
c c 



-At 



Proof. We calculate that 
d 



dt 



A 



-At 



dt 



Kvn{t,(t))-\e-^'Kvn{t,4'). 



Using \vn\ < \, this implies that 

d 



(5.1) 



dt 



JVn{t,(j)) 



<</> + e-^*fl + - + -) + 
c c 



e-^'j^Kvn{tA) 



Taking a=(l-|-^)e '^^-l-^, it therefore suffices to bound the last term on the right hand side 
above. So, 



e-^'^Kvn{t,. 
dt 



d e~^^^^ 
e'^'j^ I Vn{j{t,(t>,z))^^dz 



(5.2) 



-At 



^6 II Vn 1 1 Lip 



J —00 

dj e-^'/2 



2tt 

2 



v'^{j{t,(l),z)) 



/•oo 


dj e 


-^2/2 


/ — 00 


dt ^ 





dz, 



dz 



with the exchange of derivatives and integrals in the second equality justified by the fact that v'^ 
is bounded, established in Lemma 5.5 We now examine more closely the integrand in the last line 
above. We calculate that 



(5.3) 
(5.4) 
(5.5) 



dQ_ 
dt 



^azt'^/^ + (A - aV2)^ exp |az\/t + {X - a^/2)t^ 
+ A exp ^azVi + {X - a^/2)t| 

exp < I A -I- 



J.-3/2 I 2 2j.-2 \ I / > 02 

azut ' -I — a u t 



u 



2t 



du, 



and here the first term above came from differentiating the first term of j, and the second and third 



terms came from differentiating the second term of j. We label these terms in (5.3) (^1), (^2)) (^3)- 
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Then 
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poo 




J — oo 

POO 


< e^V 


J 


= e^V 


/I 

-at 



azt-^'^ + (A - aV2) 



1 

-at 
2 



1/2 



z — a> 



Vt\ + a) e-^^-"^)'/^/^^^^ 



+ A^ 



with Ci a universal constant equal arising from the expectation of the absolute value of a standard 
normal r.v. The second term can be treated similarly, yielding 



\{A2)\e-' '^/V2^dz < Xe 



For the third term, we have, using Fubini's Theorem for the first inequality, 



t l-OO 



au 
au 



'27r 

g-(z-au/v^)2/2 



27r 



-1 ft roo 

- A^ar3/2 / ^e"« 
2 Jo 

= X-at-^/'^Ci / ue^'^du 
This has absolute value less than or equal to Xhat~^/'^Ci^^ = Xlat^^^'^Ci^ . 



dzdu 
dzdu 



Plugging these three estimates into (5.2), we obtain: 



-xt 



dt 



KVnitA) 



^ G II Vn 1 1 ijip 



dj e-^'/^ 
dt J2tt 



dz 



<e-^'\\u 



'n 1 1 Lip 



l^e^V (^ai-'/'Ci + a) + Xe'' + A^at-i/^Ci^ j 



(5.6) 



b\\Vn\\Lipt 



-1/2 



with b= {(j) (laCi + A) + A + ^aCi) . 



□ 



Using the Holder continuity established above, we can do a trivial discretization to find e-optimal 
times. 



Corollary 5.7. Fix cf) > 0. For < cf) < 4>, one may find t*((/>, e) such that Jvn{t*{4>,e),(p) < 
^mi^^ Jvn{t, 4>) + e by making ||i^n||Lip • O(^) evaluations of Jvn{-, 4>)- 

Proof. Discretize [0, T] into equally spaced points ti, . . . , ^at, where = \^~\ , and M is derived 



from the Holder constant established in Lemma 5.6 for example, we may take M = (f>+a+b\\vn\\Lip, 
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with a = 1 + ^ + ^ and (|aCi) + A + ^Ci). Then, choose t*{<f>, e) G argmin Jvn{ti, (j))- By 

l<i<N 

emma 15.61 



min Jvniti.d)— min Jvnit, 

l<i<n 0<t<T 



< e, so t*(e, (p) must be e-optimal. 



□ 



We will uniformly aproximate Vn by a function but we do not know a priori what Lipschitz 
properties the approximation has, only that it is close to Vn-i- Therefore we need Corollary 



5.9 and Lemma 5.8 to estimate JoVr, 



Lemma 5.8. Suppose that \\wi—W2\\l°° < e. Then\\JoWi—JoW2\\L°° < e and\Jwi{t, cp) — Jw2{t, < 
e for all t, </> > 0. 

Proof. The proof follows by noticing that 



Jwi{t, (f)) - Jw2{t, 



-At 



[wi{j{t, z, (P)) - W2{j{t, z, (j)))] e-' /yV2^dz 



which is bounded in size by e for all t and 



□ 



Corollary 5.9. Suppose that w : M-|- M is a function such that \ \w — Vn\\L°° < ci- Fix (p > 0. 

For < (p < (p, one may find t**{(p, e) such that 



Jw{t**{(p,e),(p) — niin^ Jvn{t,(l)) 
by making \ \vn\\Lip " O(^) evaluations of Jw{-,(p). 
Proof. Perform the same discretization as in Corollary 



< e + 3ei 



5.9 



and let t**{(p,€) £ argmin Jw{ti,(j)). 

l<i<N 



Since — fn||L°° < ei, Lemma 5.8 implies that 
\Jw{t, <p) — Jvn{t, < ei for any t,(p >0. then Lemma 5.6 implies that 

Vn{t**i(p,e)) - inf Jvn{tA) < e + 2ei . 

0<t<T 

Using \\w — Vn\\L°° < ^1 again, 



Therefore, 



\Jvn{t**{<P,e))-Jw{t**{<P,e))\ <ei. 
Jw{t**{(p, e)) - ^ mf^ Jvn{t, (p) < e + 3ei. 



□ 



5.3. Approximating jQVnii') over all (p, for Fixed n. 



Lemma 5.10. Fix n > 0, and suppose that w : M+ — )• M satisfies \\w — Vn\\L°° < ei- Then 
using \ \vn\\Lip\\vn+i\\LipO [-^) evaluations of Jw{t,(f)), we can construct a function Jqw such that 
\\Jovu{(t>) - Jov„((/))||loc < e + 3ei. 
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Proof. Following Section 4.4 of [22], the function Vc 



can construct 6 such that Vr 



0. By Proposition 



can be explicitly computed, and in fact we 
'Vn{4') ^ ^^c(i;^), and Vn is increasing and 



3.1 



nonnegative, so it follows that Vn{(p) = for (j) > (p for all n > 0. Therefore, we set Jow{(p) = 



< 



Discretize [0, 0] into R 



\\v. 



+ 1 1 \Lip 



for 4> > 4>. So from now on, we assume that < 
points 01, ... , (j)R, with (pi = and (pR = (p. Using Corollary 5.9 for each i, we may, given w, in 
IbnlUipO (^) function evaluations calculate jQw{(pi) such that \ jQw{(pi) — JQVn{<pi)\ < e + 3ei for 
1 <i < R. For (pi < (p < (pi+i, 1 < i < i? — 1, set Jow{(p) = Jow{(pi). We have, for (pi < (p < (pi+i, 

\Jow{(p) - JoVn{(p)\ = \ Jow{(pi) - JoVn{^)\ 

< \ Jow{(pi) - JoVn{4>i)\ + \ JoVn{(pi) " JoVn{(p)\ 

< (e + 3ei) + e, 

where the second e term above is derived from the Lipschitzness of JoVn = Vn+i, established 
Since each point i requires ||fn||LipO (^) function evaluations, computing the 



in Lemma 



5.5 



approximations for all R 



"n+i||L»p points requires ||fn||Ljp||wn+i||LjpO (Jj) function evaluations. 

□ 



5.4. Approximating Vn{(p), for all < n < A^. 

Proof of Proposition\5.l\ The function vq{(P) may be computed analytically. According to Lemma 



5.10 we may compute a function vi{(p) such that 

\\vi{(p) - vi{(P)\\l° 



< 



N 



in ll'WollLjpll^^illLipO 
satisfying 



f3 



function evaluations. Applying Lemma 



5.10 



again, we construct V2{(p) 



V2{(p) - JoVi{(p) 

V2(4>) 



m \\Vl\\Lip\\V2\\Lip 



o 



function evaluations. Arguing inductively in this way, we see that we 



may compute vn{<P) satisfying 



\vn{(P) - VN 



< 



Ne 



m 



O 



iV3 



N-1 

^ ^ I I liipl 1 liip 

i=0 



function evaluations. By Lemma 5.5, ||ui||Lip < iT. Therefore 
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N^\^ ^/jvn 



i=0 



N-1 



g3 J A 



i=0 



< O ( — J N^T 



6. Numerical Results for the Lump Sum n-OesERVATiON Problem 
6.1. Comparison to the Continuous Value Function. 



□ 




Number of Observations 



20 30 40 

Odds Ratio of Disruption 



Figure 6.1. Value function Vn{<j)) for < n < 9, and continuous observation value 
function Vc{(l)), A = .1, c = .01, a = 1. 



As expected, the value functions Vn{4>) are all concave and increasing, and between 



100 and 



0. Furthermore, as n increases, the value functions decrease. From Figure 6.1 , it is not immediately 
obvious whether lim Vn = Vc, although the results of Section |3| prove that this is the case. In any 

n— ^oo II 

case, the convergence rate is quite slow, as demonstrated by Table |6.1 
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0.25- 



Number of Observations 



Prior Probability of Disruption 



Figure 6.2. Bayesian Risk Associated with Total Number of Observations, A = .1, 
c = .01, a = 1. 



Table 6.1. Effect of Observation Size on Value Functions at 







Observations (n) 


Vn{0) 


Vc{0) 


t;„(0) - Vc{0) 


'°^tg?i;r"-i°g(^°w-^-w),n>i 





-76.021 


-98.237 


22.216 




1 


-82.586 


-98.237 


15.651 


-.505 


2 


-85.755 


-98.237 


12.482 


-.525 


3 


-87.410 


-98.237 


10.827 


-.518 


4 


-88.392 


-98.237 


9.845 


-.506 


5 


-89.024 


-98.237 


9.213 


-.491 


6 


-89.455 


-98.237 


8.782 


-.477 


7 


-89.762 


-98.237 


8.475 


-.463 


8 


-89.990 


-98.237 


8.247 


-.451 


9 


-90.163 


-98.237 


8.074 


-.440 


10 


-90.299 


-98.237 


7.938 


-.429 



6.2. Comparison to Dayanik's Discrete Observation Model. In this subsection, we compare 
the value functions of the lump sum n-observation problem with those found in Dayanik's model of 
discrete observation, [14] . More precisely, we consider models of one or five total observation rights. 



and specify fixed time intervals at which observations will be made. In Figures 6.3 and |6.4[ it is not 
surprising that the value function from our n observation problem is smallest, but the efficiency gap 
can be quite large, especially for higher values of (p when it can be crucial to make an observation 
quickly. Furthermore, we can see that the value functions associated to fixed observation schedules 
have widely varying performance on different levels of (p, and one which performs well for one value 
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of (j) may do quite poorly at another. Therefore, it is hard to achieve good performance using fixed 
observation strategies. This should not be surprising: our value function is the concave hull of the 
value functions corresponding to deterministic observation schedules. The difference is magnified 
with more observations, as flexibility becomes more important. 
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Figure 6.3. Value Functions for One Obsevation: At Two, Five, and Ten Seconds, 
and Chosen Adaptively, A = .l,c = .01,a = l. 
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Figure 6.4. Value Functions for Five Obsevations: At Two, Five, and Ten Seconds 
Intervals, and Chosen Adaptively, A = .l, c=.01, a = l. 
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Depiction of Observation Boundaries. 



.1 Observation Boundary forvO S Observation Boundary for v3 




Time Since Last Observation Time Since Last Observation 



o Observation Boundary for v1 o Observation Boundary for v4 
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Figure 6.5. Observation Boundaries, A = .1, c = .01, a = 1 
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Figure 6.6. Observation Boundaries, A = .l,c=.01,a = l. 
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The observation boundary for vq is of course identicaUy equal to a horizontal line at 10 = ^. 
Without any observations, the posterior process is perpetually increasing, and so the observation 
boundary (which is really a stopping boundary here) should always stop when (p is equal to ^ . Note 
furthermore that the boundary does not depend on the time since the last observation: since there 
will never be any more observations made, there is time homogeneity. 

In the rest of the observation boundaries, we notice two general trends: first, the curves are 
decreasing in n for large values of time, and second, they are increasing in n for small values of 
time. The former phenomenon reflects the idea that if an agent has more observations, than he 
should be more willing to use them, which corresponds to the barrier being easier to get to, and 
hence lower. At small time values, however, the barriers are increasing. This reflects the fact that 
as one has more observations, one is less willing to "give up" and stop. For example, when one has 
only a single observation and the odds process is above 20, an optimally acting agent knows that 
he only hurts himself by waiting, and so will observe immediately at time zero (which is equivalent 
to stopping the game). With more observations, however, the agent is willing to wait a little bit 
and see how things will go, and for this reason, the curves increase at small times. 

A natural question is whether, as the observations increase, do the curves tend to infinity for 
very small values of time? In fact, the observation boundaries are uniformly bounded for all n 
and t. One may deduce this fact by comparing the discrete observation value functions with the 
continuous observation value function. In Figure |6.1[ one sees that the continuous value function 
is zero for </> > (here (j) ~ 55, and (j) can be explicitly computed: it is the optimal threshhold level 
for stopping in the continuous observation problem, ). This implies that every discrete observation 
value function is also zero for (p > (p. Therefore, one always wants to observe immediately at such 



(f) values. It follows then that all observation boundaries from Figures 6.5 and 6.6 will be bounded 
from above by (j). 

7. Numerics for the Stochastic Arrival Rate Problem 

7.1. A Heuristic Algorithm for the Stochastic Arrival Rate n-Observation Problem. 

Here we outline a computational algorithm for solving the stochastic arrival rate problem. The 
infinite horizon problem is a limiting case of this one. As discussed before, the lump sum n- 
observation problem is essentially embedded into this problem, so it should be of no surprise that 
this problem must be solved first. 

(1) Fix the total number of observations A^. Discretize (p into (pQ = 0,(pi, ... ,(pj = (p as before, 
and set all value functions equal to zero for (p> (p. Compute the (approximations to) value 
functions ■{Q.,(p),Q < j < A^, as in the previous section, as well as the optimal times 

(2) We have computed 9^j(0, </)), the approximation to ■{^^(p). For y,t > such that 
{y + t,^{t,(P)) E A^, define Ay + t,f{t,(P)) by 
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(7.1) e-^^y+'^v^^^{y + t,ipit,^)) = 

\ e-^(y+'^KdS^^^,{y + t,ipi-y,^)) if y + t > 0)) 

(3) Fix Discretize time into to = 0,ti, . . . ,tK = T, for an appropriately chosen upper bound 



tj. Let i^^x N-ii^j) minimizing tj. 

(4) Interpolate to find a function Vj!^_^ ]^_^{0,(j)) which approximates v^_-^^ ]^_^{0, (j)), and a 
stopping boundary 

(5) Fix 4>j. As in Step 3, compute v^_i N-2i^^ 'f'j) minimizing 
J^{v^-i^N~i^^N,N-2)i'ti^^^4>j) over the U. Let *^^i,Ar_2(</'i) be the minimizing U. 

(6) Interpolate to find a function v^_-^^ j^_2{0, 4>) which approximates v^_^ ^_2{0,(f>), and an 
observation boundary 7v-2('^)- 

(7) Repeat Steps 5 and 6 to compute v^_-^ ^(0, </>) and t*j^^i j{4>) fov < j < N — 2. 

(8) We now need to repeat the analog of Step 2. For y,t > such that (y + t,ip{t,(j))) G A^, 
inductively define j{y + t, (p{t, (p), < j < N — 2, by 



T, as in Lemma 5.4 Compute v^_^ j^_^{0,(pj) by minimizing J^v^ j^_^{ti,0, over the 



e 



(0, v'C-y, </-)) if y + t < (■/^(-j;, 

-l{u<a+t}e~^"^jy_i,j+i(?/ + «. V'C": <^)) j 
g-A(y+t)g-M(y+t) js:pjV_^ + t, <p{-y, 0)) if J/ + t > t^^i,, (V'C-?/, 0)) 

(9) Repeat steps 3 through 8 for each < n < — 2, computing v^j, < j < n and their 
associated optimal times i^'" ((/>). 

7.2. Discussion of the Heuristic. 

1. The formula in Step (2) comes from a dynamic programming principle. In a simplified 
version (with y = 0), the dynamic programming principle says that for t < tj^j {(/)), 

in other words, if it is not optimal to make an observation before time t, then by waiting 
until time t no utility is lost, and the only difference between the value functions at the 
two times is the exponential discounting and the running cost lost between them. On the 
other hand, for all t > (</>), we assume that it is optimal to immediately observe, hence 



the term e ^^Kv^ -.^{tjcj)) in (7.1), describing the expected value after an observation is 



made. This step is a heuristic because we have not shown that the optimal observation 
behavior has this simple strategy. It is in theory possible, although unlikely in practice, 
that, when starting at t = the optimal observation time is 'Pjf^j{(t>), but when starting at 
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some ti > t*jf^j{(t>), the optimal observation time is not ti, but some other t2 > ti. Numerical 
evidence suggests that this is not the case, but we do not have a proof of this fact. 

2. In Step 3, The Jq operator is applied in the case when the agent has no spare observation 
rights. If he must wait to receive an observation, then even in that first instant when he 
receives this right, a positive amount of time has passed since the last observation was 
made. Therefore, we need information about the value function, i.e. ^_-^^{-, •), when its 
first argument is positive: this explains the necessity of Step 2. Similar considerations apply 
to the calculation in Step 5. 

3. The derivation of Step 8 is similar to that of Step 2, except that whereas in Step 2, we 
took the dynamic programming principle with all observation rights received, here we use 
the dynamic programming principle when there are still observation rights receiving. For 
example, dynamic programming implies that for t < t*j^1ij{4')t 

VN-i,j{^, 0) = Me-^ ( er^' \^{r, </>) - ^ J dr 

+ l{t<«}e"^*t^i^_ij(t, <^(i, ^)) + l{„<t}e-^"<^^ («, ip{u, (^)) j du. 

Prom this equation, we can solve for v^_^ j{t, ip{t, 4>)) to obtain the formula used in Step 8. 

4. In Step 3, the minimum of a function is calculated by an exhaustive search on grid points. 
Numerical evidence suggests that J^v^ j^_i{t,0,(f)) is actually convex as a function of t, 
which would allow for much more efficient ways of finding its minimum. The same holds 
true for the minimization in Step 5. We currently do not have analytic proofs of these facts. 
We note that this reasoning can additionally be applied to the minimization of Jvn{-, (f>) in 
the lump sum n-observation problem. 
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7.3. Numerical Results for the Stochastic Arrival Rate n-Observation Problem. 
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Figure 7.1. vl^-^^{(p),vl^Q{(p), and VQ ^^cj)) for different arrival rates /u; A = .1, c = .01, 
a = l. 



Concerning Figure 



7.1 



we make a few basic obsevations. First, it should be clear that vl^, 
corresponding to the case where the single observation right has been used, is the worst-performing 
value function, and vJq, corresponding to the case where the observation right has been received 
but not used, is the best-performing. We also expect Vqq, as the arrival parameter // varies, to 
interpolate between these two extreme curves, so that Vq q resembles vl q when fi is large, and t>| ^ 
when /i is small. Furthermore, the gap between Vq q and u j q is smallest when (p is small. This 
reflects the fact that when (p is small, an optimally acting agent, even if he had an observation right 
in hand, would wait to exercise it. As a consequence, having to wait to receive such a right is a 
less stringent constraint than when (j) is large, in which case it is important to make an observation 
relatively quickly. 

8. Proofs from Section [2] 

Proof of Proposition \2.3[ Let = {ipi, . . . ,ipn} £ and let r G . Supposing that r > tpk, 
we will show how to modify ^ to yield an observation strategy ^' under which r is a F*-stopping 
time that does not stop between ipk and ipk+i- By inductively following the same procedure, this 
allows us to construct an observation strategy ^' such that r G 7^*' and [JJ^ e~^* — ^) dt\ = 



dt 



. This will establish the lemma. 
The basic method is to just add in an observation whenever r stops between observations; if a stop 
is made between observat ions , there are always "spare observations". Let A = {ipf^ < r < V'fc+i}- 
According to Proposition 2.2, A G J"^^, or ^ G fi ({X^-,il;i}^^.^A. Define -^i = Vi, • • • , V^fc = V'fc, 
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V'fe+i = lAT+lA'^ipk+1, and for k+2 < j < n, ipj = l^V'j-i+lA^V'j- With Aea ({^Vi' V'j}i<i<fe) = 
a { \ Xj j'tpi} ], the following claim will imply that for each A; + 1 < 7 < n, 

\l 'Vi J l<i<fc J 

e m (t{X^^,. . . . . . , V'i-i). 

This fact is obvious for j = 1, . . . , k, as i/^j = tpj for j < k. Set * = {^1, . . . , V'n}- 

Claim 1. Let Letk + l<j<n. Let X G and let Y e T^. . Then [Af^X\VJ [A"" n Y] G j| . 

Proof Of Claim,. Write Ix = x {{X^^.tlH : 1 < i < j - 1}), 

ly = y ({^Vi' V'i : 1 < « < j}), 1a = a V'i}i<i<fe)' where y, and a are all Borel functions 

with respective domains M^(-^~^),M^-^, and M^*^. 
Then 

= lAx({X^„Vi:l<i<i-l}) 
+lAcy({X^„Vi : 1 <i< j}) 
= Iax (X^i, ■^i, {l^^Vi-i + l^V'i-i + lA'^V'i : 3 < z < j}) 

{l^^Vi-i + ^A'^X^., lAtpi~l + lA'^V'i : 3 < z < j}) 
= a ({-'^^Vi''^«}i<i<fc) ^ (^Vi'V'i,{lA-'^Vi-i + ^AX^^,lA1pi-l + Uc^i : 3 < i < j}) 

+ (1 - o ({^V'i''^i}i<i<fc)) V'l, + U'=^V2! + 1a=V'2, 

□ 

Having concluded the proof of the claim, we have shown that the observation strategy * is 
admissible, or ^ G D". By construction, r does not stop between i/^k and tpk+i- 

We have to check that r is a F*-stopping time. Let t > 0. Then {r < G if and only if 
{t <t} n {il^j < t} G T- for each I < j < n. This is clear for 1 < _7 < A; as ijjj = tpj ioi 1 < j < k 
and T is a F*-stopping time. For j = A; + 1, we work on A and A'^ separately. We must show 

{r<t}n{^fc+i <t} G 

Invoking the claim, it is suffcient to show that 

(8.1) {r<t}n {ifk+i <t}nAeJ^^^ 

and 

(8.2) {r<t}n {ifk+i <t}nA'^e ^1^^ . 
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To show (8.1 ), note that on the set A, V'fc+i = t, and so {r < t} n {V'fc+i < t} n A = {t < t} H A. 
Since r is a F*-stopping time, greater than or equal to ipk, it follows that {t < t} = {t < t}r\{'ipk ^ 
t} G To show (8.2), note that on the set A'^, ipk+i = V'fc+i) so {r < t} n {ipk+i < t} r\ A'^ = 

{r < t} n {ipk+i < t} n A'^ G -^*fe+i' '^o^^S that r is a F* stopping time and A G J"*^. 
Now, fix J > k + 2. We may write 

{r < t} n {4)j <t}= Ar\{T <t}r\ {V^j-i < t} u yl'' n {r < t} n {tpj < t} 

Since r is a F"^-stopping time, we know that {r < t} n {ipj^i < t} G and {r < t} n {t/'j < 

t] G J"!^.. Therefore, {r < t} n {^j < t} G Jj,, again by Claim |l| 



Finally, note that E'l' [j^ e"^* ($f - ^) dt] = E^ yj^ e"^* (^$f 
on the random time interval [0,r). 



because = <I>* a.s. 



□ 



Proof of Proposition \2.4\ Fix n > 0. We proceed by backwards induction, so consider the base case 
k = n, and take = {ipi, ■ ■ ■ , V'JJ} £ i'"- We will prove equality here. We must show that 



ess inf ess inf E 



ess inf E 



the second inequality following from the fact that we can observe a total of n times, so that D"'(^") 
is a singleton, and equal to 
Recall 



(8.3) 
A 

According to Proposition 



f* 



c + A 



VO. 



.c(0 + l) 

A simple calculation confirms that JoO(0) = JO{t* {(/)), (j)). 

there is a one-to-one correspondence between {r G T*" : t > 
and the set {V'^J + i?„ : i?„ > 0, i?„ G So, take r* = V'n + ^SC^^^ ^ ^ T'^" ■ ^ > CI- 

Thus, 



ess inf E 

rer*",T>V'Ji 



A 



dt\j='^„ 



According to (2.1 ), the process $f " is increasing for t > and if ^{^f") > 0, then '0^-|-tQ(<I>*r) 
is the unique time t > when ^f" — ^ changes sign from negative to positive. If tQ{^f") = 0, 
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then is always greater than A/c. Therefore, 
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ess inf E 



> E 



f 



I^n \ C 

the inequaUty holding at the pointwise level. It therefore follows that 7"(^'") = vo(^*n)- 

Now, for the inductive step, suppose that 7^ > Vn-k-i ( ) for all = {'0f'^^}i<j<fe+i G 

Dfe+1. Let '^^ = {ipi}i<i<k e O^. We wish to show that 

Let ^ G D"(**^), and write ^' = {V'l , . . . , ■0^, V'fe+ij ■ • • > V'n}- Without loss of generality, we assume 
that V'fe+i > V'fe- 
Then 



ess inf E 



min < 0, ess inf_ E 



= min I 0, 



+ essinf_ 

Te7^,T>i/>fe+i 



/•V-fc+i— i/'fc / 



> min < 0. 



/- 



\j k 

by the inductive hypothesis, where we have used the fact that ipk+i is J^jk -measurable as well as 



the deterministic dynamics of <I> in between jumps. Next 



min < 0, 



min < 0, 



e I 



dt 



Vn-k-l 









I 77* 



= min-|0,y e"-^* ( V5 



= min |o, Jvn-k-i (V'fe+i - V'fc, } 
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noting that tpk+i — V'fc > and that 
This estabhshes that 



^k+i 



has standard normal distribution, independent of 



• Finahy, min <j 0, Jvn-k-i ( V'fc+i 



> JqV. 



essinf E 



n—k—l 



Vn-k ^, 



Taking the infimum over ah ^ G D"(^'''), we obtain that 7^(^''') > v^-k (^'% 



□ 



7fc(^'') < Wn-fcl*^!^*")- Proposition 
base case has aheady been estabhshed in 



Proof. The " > " inequahty has been estabhshed in Proposition 2.4, and it suffices to show that 

we proceed by reverse induction, and note that the 
Therefore, we assume that Jk^ii"^^^^) = 



2.4 



roposition 



2.4 



Vn-k-i ( ) for all ^^'+1 G D''+\ Let G D^. We wish to show that 7^(^''=) < Vn-k (^%' 



Recall the functions 



= min{s > : Jvn^kis, (/>) < JoVn-k{4>)}, 



and the stopping times ip'^'^^ ~ V'fc + ^n-fc (^^^ ) ■ ^o^e that we can use a minimum above instead 
of an infimum because Jvn-k{s, (p) is lower semi-continuous in s, as Vn-k is nonpositive. Write 



Then 

-(/^(vj/*^) = essinf 



ess inf E 



'At: 



A 



< essinf essinf E 



min < 0, 



ess inf ess inf 



= min < 0, i? 



min < 0, E 



-At 







I 77* 



with the last equality following from the inductive hypothesis and the deterministic evolution of 

T k 

$ in between jumps. It now follows, using an argument similar to that at the end of the proof 
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of Proposition 2.4, that 



min \0,E 



(T k 

Vn-k (^5) , 



with the first equahty above by definition of V'fc+i and h^_j^. It follows then that 
and the equality now follows. 



□ 



Proof of Lemma 2.6 We claim that h^^_i^{(j)) is lower semi-continuous. This would imply that 
is Borel- measurable, which will in turn imply that ip^^^ is a stopping time. 
Let 64 6 00 in M_|_, and let Sj — h^__i^{4'i) ■ Note that JoVn—ki') is bounded, and that 

lim ini Jvn-k{s, (j)) = +00. 

s— 5>oo 0>O 

Therefore, we may assume that the sequence {sj}j>o is bounded. It follows then that liminf Sj = 
Soo < 00. It is straightforward to see that Jvn-ki', •) and JoUn-fc(") are continuous in their argu- 
ments. Therefore, 



JVr, 



< liminf Jv^-kisi, (t>i) 

< lim JoVn-k{(pi) + e 

J— >oo 

= JoVn-k{(t>OD) + e- 



Thus, 



K_k{(l)oo) < Sc 



liminf Si 
liminf h[ 



establishing lower semi-continuity. 



□ 



9. Proofs from Section [3] 



9.1. Estimates for the Second Moment of <I>". 

Lemma 9.1. Let Z he a standard normal random variable. Then fort > and (t)'>Q, E[j{t^ Z, 
e^*((/>+ 1) - 1. 

Proof. We may write 
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The integral of the first term is calculated by completing the square and equals e'^V- The integral 
of the second term is calculated by switching the order of integration and completing the square, 
yielding e^'' — 1. □ 

Corollary 9.2. Let n and k be positive integers. Then 



E 



^ k K fc-1 



I- Ti n 



en $^1 +1 -1. 



Proof. Apply Lemma 
dent of . 



9.1 



with t = ^, using the fact that = j ( ^, Z^, <I>^ | , with Zk indepen- 

□ 



Lemma 9.3. Let Z be a standard normal random variable. Then for (j) > and t > 0, 

E[jit, Z, 0)2] < (/>2e2^*+-'* + 2<Pe^'^-^ (^^xt+ah _ + (^gAt _ 

Proof. We start by expanding E [j{t, Z, 0)2] into three terms: 
E[j{t,Z,4>f] 

^2g2a.Vt+2(A-aV2)tg-.V2/y(2^)dz i (1) 



oo 



+ f (j[|*Ae^"-^^"-^"^dn) (^l\e'-^^^-^-'dw^ e-^/V^/2^cfz} (3) 

We calculate each of these terms seprately. (1) is the simplest, and by completing the square, we 
can calculate its value to be (fp'e^^^e'^^^ . We have 

(2) = 20 / Ae^*e^"e°'" 







= 20e^* / Ae^"e"'"dn 
Jo 

= 20e^*-^fe^*+-'*-l 
A + a"^ V 

Now, the third term we cannot calculate exactly, but we give an upper bound for it which will 
be good enough: 

(3) = r [\e'-e'-e^ ( H e<'-'^^-'^T /V2^dz) dudw 

Jo Jo \J~oo J 

* f Xe^^e^^e'^dudw 
Jo 
t ft 



< I I Xe^''e^'"e''''dudw 



JO 



where in the inequality above, we have the used the fact that uw < t^ for u,vj G [0,t]. Combining 
(1), (2), and (3), we deduce the lemma. □ 
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Corollary 9.4. Let n and k be positive integers. Then 
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E 



A 



A + a2 



- 1 +e- e- - 1 



Proof. Apply Lemma 9.3 as Lemma 9.1 is used in the proof of Corollary 9.2 



□ 



Since we are interested in the limit as n — >• oo, we we also set down asymptotic versions of Lemma 



9.3 and Corollary 9.4 



Lemma 9.5. Let (/) >0, and Z a standard normal random variable. Then as t ^0, 
E [j{t, Z, 0)2] = 02(1 + 2\t + ah + 0{t^)) + (P{2Xt + 0{t^)) + 0{t^). 
Proof. By examining the proof of Lemma 9.3, we can see that E\^j{t,Z,(j)f''\^ = 



2(f)e^^ A+a^ ^gAt+Q t _ ij ^ pj^g positive third term, which can be bounded from above by * (e^* — l)' 
Note that this term 

^ah I^^Xt ^^i^^2^^ 0{t^)){Xt + 0(t2))2 

= 0{t'). 

So, we are left with the first two terms. We have 

^2^2Xt^aH = + 2Xt + 0{t^)){l + ah + 0{t^)) 
= 02(l + 2At + a2t + O(t2)), 

and 

2</'e^* ('gAi+a^t _ ^ 20(1 + \t + 0{t^))j^{\t + ah + Oit")) 



X + a^ 



20(i + O(t2)). 



Corollary 9.6. Let k be a positive integer. As n ^ co, 



□ 



E 



-pn 



„ 2/ 2\ fl 

$"-1 ) + — + — 
n n 



O 



9.2. Proving Proposition |3.8 by Establishing the Conditions of Proposition 3.6| , We 



verify the six conditions of Proposition 3.6 separately. First note that Condition (a) is satisfied by 
construction. We will defer (6) until last. 

Conditions (c),(e). First, we will construct the process N^. This is essentially done by Doob 
Decomposition. We set A'^q = 0. First, we will define A^" at grid points {^, ^, . . .}, and then extend 
to all of M+. We construct A^"" on the grid points inductively: for A; > 1, 

k 



(9.1) 



A^^ - A^Li = E 



E 



^1 - 



n 



b{<i>^)ds\rLi 



A 1 + i>ri i^Li 
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using b{x) = A(l + x), as well as the fact that <I>" = for < s < ^ by construction. Now 

71 

by Corollary |9.4[ 



E 



en +1-1 



(9.2) 



1 + - + ^ + • 
n 

n V f^-^ 



Therefore, plugging (|9.2|) in (|9.1|), 



(9.3) 



n 

1 



+ 



+ 1 



^Li +1 

^"-1 + 1 



+1 

5^1+1 

n 

1 - $'^-1 



1 



n 



1 
1. 

A ( 1 + 



This defines A^"- on aU grid points. Next, for <t<^ 



k-1 



n 



b{^'l)ds\Fl^ 



t\\[l + 



using as before the forms of h{x) and in between grid points. 

We have now defined NJ^ for all t > 0, and by construction, (6) is satisfied. The goal now is to 
show that Condition (e) is satisfied by the A'^^'s as n — >• oo. We start with some observations about 
the process N^. Recall the fixed -L > from Proposition |3.6[ W^e let kmax — kmax{f^) — Ln. 
(1) For all n, the sequence A^" = {Nq, N^,N^, . . .}, is increasing: Note that from (9.3) 



n n 

i2 x3 



A" A^ 



+ 



^"-1 + 1 > 



As a consequence of this fact, the maximum of E [A^j-] over all of its stopping times T is 



equal to E 



N kmax 



(9.4) 



(2) For any k, A^f - = - t) A ( 1 + ) for ^ < t < ^. This is a negative 

term whose magnitude is maximized when t approaches ^. Furthermore, A^" — A^"_i = 

n n 

— ^ I 1 + I . Since A^" is always nonnegative, this implies that 

y n J n 

A 



inf A'^," > min 

0<t<L l<k<k,nax 



n 



1 + 
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Note that ^^li '■ < k < kmaxj is a submartingale, as seen in (9.2), and so its negative is a 
supermartingale. Therefore, by Doob's inequahty, 



(9.5) E 

We iteratively apply Corollary 



A 



mm ( 1 + 



<2\,E 



9.4 



(9.6) 



E 



to estimate the size of , deducing that 

n 

2" 



in 



In particular, for all n and for k less than or equal to kmaxin) = Ln and fixed (j), the above 



quantity is uniformly bounded over all k. It follows from (9.4), ( |9.5[ ), and (9.6), therefore, that 
ll^mf^iVf 11^2 = O (^). We find, therefore, that 



sup E 



E 



+ 



It remains to control the size of this last term. Recah from that iV^-iV^i = O {^) [ + 1 

Iterating this formula over 1 < A; < kmax, we have 



using kmax = Ln. Squaring both sides of this equation and taking expectations, we deduce by (9.6) 
and Holder's Inequality that 

E 



^ n 

= O (1) (fc„„(n))=e=^<"*"") + O (1) fc,„„(n).''P^+"'' + O (1) 



1 



This establishes Condition (e). 

We next address Conditions (d) and (/). 



□ 
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Conditions (d) and (/). First, we will construct the process A/""- The procedure mimicks the one 



in the construction of N[^. We write 



-5^l = (M^'- te'' 



(9.7) 



Ml 



using the fact that a{x) = a^x^, and that = for ^ < s < ^. We set Mq = 0. We first 

n 

define AA" on the grid points ^, . . .}. Define 



on QTL I 



(9.8) 



n n 
k 



E 



fc-1 \ 2 

" 6(i;)<is - wtj I 1^ 



n \ ^ 



We can expand the first term in (9.8) as 

k-l 



E 



or 



E 



- j 6($^)rfs - iV^l J + - j - \ K^^^)ds - (Af^ - iVLi ) 



K fc-l 



This is designed to cancel with the second term in (9.8). After some algebraic manipulation, we 
arrive at 



+ E 



b{^'l)ds - N'Li E 



In the first term of the right hand side above, the conditional expectation is zero, by definition of 
N"'. So we have 
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E 



- ( + - ( 1 + 



We will now expand this term above. Recall the useful facts that E 
1, and that A^" is -measurable. We have 



U' fc-i 



* fc K fc-i 



n 71 



+ 1 



n 



(9.9) 



2 ( ^-^.i + - ( 1 + ) + ( iV^ - iV?-i ) ) E 
n 



\ \ 2 2 



The second term on the right hand side of (9.9) is equal to 
(9.10) 



2 ( + - (l + cD^i) + (m - iVL,) ](e^{ cI>^l + 1 ) - 1 ) . 

n Til \ n ^ ^ n n 



According to Equations ( [9l3| and ( |9l6l ), ( A^^ - A^^ ) = ^ ( ^Izil + ^ ) ' uniformly over ah 



^ < km.ax{n). Then (9.10) becomes 
-2 



c^^i + - fl + ^^l") + O ( 4 I ( + 1 



n 



en «>^i +1-1 



+ - f + + o f ^ ) ( ■J'^.-i + 1 



As we can see, this term partially cancels with the third term in Equation (9.9). In doing so, we 
obtain 



(9.11) 

Ml -A^Li =E 

= E 



■k-i -r - il + <I>'^-i 
n 



^"^-1 +1 



From Corollary |9.6[ we have 



E 



($1 



= 1 (1+ — + — +0 



O 



1 



and this perfectly cancels out with the second and third terms in (9.11). We are ultimately left 
with 
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Before pursuing this line of reasoning further, let us define A/"" between grid points. For < 



t < - 



using a{x) = a^x^ as well as the the fact that = <J>fc_i for < s < ^- As in the proof 

n 

of Parts (c) and (e), we may show, using the submartingality of < <l>" : /c = 0, 1, . . . > , that the 



norm 



inf - ^ ( 

0<k<kmax{ri) " 



<^ ^ll^femai; IIl^' which is O (^). Therefore, in attempting to 



establish (e), we may ignore any possible times in between grid points {0, ^, . . .}. Consequently, we 
consider the discrete-time process 

Ar'^^{AAo",AA£,...,AAL^}, 



with associated bounded stopping times T]^ , and we must show 



sup E[\Mx\] — as n -)• oo. 



According to (9.12), for any k, 



o(;^)|:((%)'+%+0- 



Therefore, noting that is always nonnegative 

n 

h 



sup 



AA^ 



;?)E((%) + ' 

j=l 



Now, since ■{ ^ is also a submartingale and kmax = Ln, it follows that 

fc>0 



E 



sup 

ax 



< O 



O 



n 



{Ln)E 



^1 



+ + 1 



□ 
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Condition (b). First, we claim that E {j{t, Z, 



0{t)(j)'^ as t — )• 0, for Z a standard normal 



random variable. Supposing that this claim is established, then applied conditionally, it entails that 



E 



n - 1^-1 



and this will establish (6), given our control over the norm of sup 

0<k<Ln 

the claim. 



So, let us address 



We have 



E 



{j{t, Z, 4>) - (t>f\ = E[j{t, Z, <A)2] - 2<t>E[j{t, Z, 4>)] + 
= 02 (1 + o{t)) - 2(P{<P + Oit)) + 



here, we have used Lemma 9.5 for the first term, and Lemma 9.1 for the second term. 



□ 



10. Proofs from Section [4] 



Proof of Proposition 4-5. The proof will be by backwards induction on j. For reference, the reader 



should see Figures 4.1 and |4.2[ we will use backwards induction on the columns in Figure 4.1 , and in 
each column, we will use backwards induction on the elements of the column. First, the base case, 
where we will prove equality. Suppose that j = n and < A; < n. Let = {ipi, ■ ■ ■ ,'0fc} £ 
As mentioned before, the Poisson process is assumed to be stopped at rjn, so that r/n+i = oo. 
Since all observation rights have arrived by the time r/j when j = n, we have remaining a total 
of n — /c observation rights, with no restrictions on when they may be used. Therefore, following 
the proof of Proposition |2.4| with slight modifications, we may establish that for each < k < n, 

We next tackle the inductive step. Suppose then that 

holds for all < < J + 1 and any ^'''^ G , on the set {V'^'^ < V{j+i)+i}- We wish to show 
that 7"jt(^''') > vjf^ (ip^ V rjj - Tp^, ^^^^ J holds for all < k < j and any G £)^, on the set 
{^fc < To establish this, we will proceed with another round of backwards induction, this time 

on k, starting from k = j. So, we fix = {V'l, . . . , V'j} S OK Let ^ = {■i/'-J, . . . , -0^+1, . . . , V^n} £ 
0^j{^^). Note that since ^ € D", it is the case that V'j > so that -i/'j V r]j = -i/'j- We have, with 
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all arguments taking place on the set {ipj < 



(10.1) 



essinf E 



dt\T 



essinf E 



+1 



{T>r)j+i} 



dt 



For each such r above, r V f/j+i is an element of which is greater than r/j+i = V'J V 



Therefore, the right hand side of (10.1) above is greater than or equal to 

r»7j + lAr 



essinf E 



-A(t7j+i-V':^) n 



which by the (initial) induction hypothesis is greater than or equal to 



(10.2) 



essinf E 



A 



dt 



+ ^{^>'7j+l}^ 



-A(7yj+i--0^) 



Now, recall that rijj^x — ip-^- is independent of J-Jj and exponentially distributed with parameter 
fj,, on the set {ip-'- < 77j+i}. Additionally, from Theorem 3.2 of |14j . it can be seen that for some 



nonnegative random random variable Rj G mJ-^- , = 1 



{Rj>vj+i-i''}' 



and rl 



{'r<'nj + l} 



becomes 

ess inf 

which is equal to 



fie 



Using the deterministic dynamics of in between observations, (10.2) 



RjAu 



du, 



^3 



ess inf ( i?,- , 0, <!>;;;; ) > Jo% 



TO„.n 



0,$ 



This implies that 



essinf E 



dt\J^^ 



>^^,(0,$ 



on the set {ipj < ?7j+i}. Taking the infimum over all ^' € l0^j{^^), we obtain that 'j^'j^^^) > 



Vj'j I 0, J on {^j < 7]j-^-l}, which establishes the base case in the second induction. 
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We now proceed to the inductive step in the second induction. Our hypothesis is that 7^;j_|_i (^^~^^^ > 
to show that for any ^''^ = {^J', . . . , V'f } € O*^, it is the case that 

So, let $ = {V'f , • • • , V'fc, V'fe+i, • • • , V'n} e £>",jt(*'')- We write, on the set {ip^ < rjj+i}, 



ess inf E 



= min < 0, ess inf E 

I Ter*,T>Vfc+iAr)j+i 



V'fc+lA%+i 



+ 



using the fact that r G 7^*, i.e. it does not stop the game between observations while there are 
remaining observation rights. This then equals 



mm 



+ essmf E 



min <^ 0, S 



ripk+iAVj+l 



+ cssmf 

i-er*,r>Vfe+iA»jj+i 



{V'fc+i<%+i 



r 



+1 



{r;j+i<V'fe+i} 



(10.3) 



> min 



-ipk+l^Vj+l 



+ essmf £^ 
+ ess inf 

rer*,T>-0fc+iA»7j+i 



1 / e-^(^-'^^''^) ( - 



dt I J\f 



{'7j+i<V'fc+i 



(it I V / 
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which we claim is equal to 



(10.4) 



mill \0,E 



+ ess inl^ E 
+ ess inf E 



{Vj+l<tpk+i} 



^k+1 



Vj+1 



I VfcVrjj 



dt J~ / 1- , , 



It is trivially true that (10.3) < (10.4), since its infimums are over a larger set of stopping times. 



We claim that the other inequality also holds. To see this, let r > V'fc+i f\ Vj+i be given. Consider 

, which is a stopping time greater than Vfc+i takes the 
The existence of such a r' implies that minimizing over 

{r G : T > tpk+i A ?7j+i} is equivalent to minimizing over {r G 7^* : r > Tpk+i}, on the set 
{'i/'fc-l-i < r]j^i}. A similar procedure may be done for stopping times on the set 1 



{ipk+i<Vj+i} {Vj+i<tl>k+i} 

same value as r on the set IrT ^ i 

{ilJk+i<Vj+i} 



establish the equality of (10.3) and (10.4) 



{V]+i<i>k+i}' 



to 



Next, conditioning the second and third terms of (|10.4|) on, respectively, J-~ and J-~ 



and using the definition of 7"^_)_;^(^), 7^+1 A;(^)' obtain 



(10.4) 



> min \0,E 



+E 
+E 



7 . ie"^^'^*+'"'^'^''^^7?fc+i(^)|-^*; 



{»?j+i<V'fc+i} 



where we have used the fact that ipk+i > rjj (so V'fc+i Vr/j = V'fc+i) for the second term, and the fact 
that we are on the set {V'fc < f/j+i} for the third term, so that V r/j+i = r/j+i. Now, applying 
the induction hypothesis, this is greater than or equal to 



min 1^0, £^ 

+E 
+E 



1 
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equal to 
(10.5) 



mill '1^0, 

+E 
+E 



IrT 



-A(Vfc+i-i/>^Vr,,) n (q ^ Ijr* 



-A(»7j+i-V'J:Vr)j)^n 



1 ~ e 

{?)j+i<i/.fe+i} 



T-.VJI 



Now we make some observations. First, on the set {ip^ < r/j+i}, Vj+i ~ V'fc V rjj is independent of 
■^**vt; ' ^'^^ conditioned on this sigma algebra, is distributed as an exponential random variable 



with parameter ^u. Second, between -i/;^ V rjj and V'fc+i ^ 'Hj+ii has deterministic dynamics 
described by (2.1 ). Third, for some nonnegative J^*^,^^ -random variable ipk+i = '^k'^'nj~^Rj,k, 

Fourth, we note that. 



1, 



^{t/.fe+i<r,,+i} - -^{i?,,fe<r,,+i-^feVr?,}' ^ud l{^^^^>^^^j} - ^{fi^- fe>,7,+i-,/,feVr,,} • 



based upon (2.1) and arguing as in Proposition 2.4 



E 



Therefore, 



(10.5) 



min •{ 0, y 



-Xu 



du 



which is equal to 



mm ■ 



We have shown that 



ess inf E 



> 
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Thus, after taking the infimum over all ^' G 0^f^{"^^), we deduce that 



as claimed. 



□ 



Proof of Proposition 4-6. As in Proposition 4.5, the proof is handled by a double backwards induc- 
tion, first on the number of observation rights received j, and then on the number of observation 
rights spent, k. The proof of the base case, j = n, corresponds to times when all possible obser- 
vation rights have been received, and so as before, this case is handled essentially identically as in 



Proposition 2.5 Therefore, we move on to the inductive step. Suppose that the result has been 
proven for j + 1 observation rights received; we will prove it for j. Now comes a second induction, 
on k, so we will take up the base case of this, and take k = j. So, let G £)■'. We have 

(10.6) -yU^n 



ess inf ess inf E 



dt\F 



(10.7) 



< E 



+ ess inf ess inf E 



V3+1 



dt\j^' 



where we have used the following facts: 

(a) The j + f** observation can not be made prior to the arrival time ^j+i of the j + 1** arrival 
time, so <I>* evolves deterministically between ipj and rjj for all ^ G iD"j(^'-^). 

(b) For any r G 7^"^ with r > f/j+i, we may construct the stopping time r = ^jl{?".<r;j+i} + 
rl|f™.>^^,^j} which agrees with r on {rj^^- > rjj^i}, and is an element of 7^* such that 
T > ipj. Therefore, the infimum over t > ipj in (10.6) can be replaced with the infimum 
over T > r/j+i in (10.7). 

After conditioning the interior of E 



dtlj^' 



we see 



that (|10.7|) is equal, by definition, to 



E 



which by the induction hypothesis is equal to 



E 



+ 1 



QUICKEST DETECTION WITH DISCRETELY CONTROLLED OBSERVATIONS 

which is equal to 



51 



flO.. 



A 



ds 



+ 1 



4-*^ 1> 



du, 



^ ~ ~^ ^Ij ylp^ j ' ^^"^ using the fact that on the set {-i/'j < 'nj+i}^ Vj+i ~ i^j is independent 
of J-'^j , and distributed as an exponential random variable with parameter /i. 

Now, (10.8) is equal to J^V^^^j {^^ j {^^^ '^^^'^^^ construction of s"^, is equal to 



Thus, we have established that 7"',(^-') < v'^- ( 0,$*^ ) on the set {^■'- < r/j+i}. In light of 



Proposition 



4.5 



these quantities are actually equal. Furthermore, since Vj'j ( 0,$*^ J < 7"j(^'-') < 



(|10.6D = fj^j. ( 0, ] , (|4.12D is established. 



We now proceed to the second inductive step. Supposing that the result has been proven for 
j observation rights received and k + 1 < j observation rights spent, we will prove the result for 
j observation rights received and k observation rights spent. So, let G O^, and let ^1''^'+^ = 
{V^, . . • , e O^.+H^''), with T^,^ = T^^ki'^''). We have, on the set {V'^ < r/^+i}. 



(10.9) j-.i^'^) 



ess inf ess inf E 



< E 



essinf_ ess inf i? 

ess inf ess inf 



where we used in the first line above the deterministic evolution of in between observations, 
and for the second and third lines, the structure of stopping times in this problem, arguing as in 



(b) above. Now, (10.9) above is equal to 
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E 



which by the induction hypotheses (first induction for the third term and second induction for the 
second term) is equal to 



(10.10) E 

+ e 



ds 



We will now argue as in (10.5) and the discussion following it. Using the fact that r/j+i — V rjj 
on the set {^/^| < is independent of J^^k^^, and exponentially distributed with parameter n, 



and noting rj^fc - 'i/'fc V r?j = 0"^^. = o";^ ( V'l; V r]j - -0^, <I>^fey^ ) , ( |10.10[ ) becomes 



fie 



+e 



-As 



which is equal to J+ (^I'^+i^fc' ''^j^fc+i) {^],k + V'fc V ry^ - -i/'f, *^*fcv»7j) ' "^^^^^^ ^y construction of o^_^, 
is equal to 



Thus, we have established that 



k 



on the set {ij:^ < Equality is now a consequence of Proposition 4.5 Examining the proof, 

we immediately deduce (4.13) as well. □ 
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Appendix A. Posterior Dyamics 



In this Appendix, we derive the recursive formula (2.1). It will be convenient to assume that all 



observations occur at deterministic times, following [T^. The reduction to this case from observa- 
tions at stopping times follows immediately from iteratively taking conditional expectations. All of 
the material in this appendix may be found in |14j . 

On some probability space (0, P), suppose that A" is a standard Brownian Motion which gains 
drift a at time 9, where P(G = 0) = vr and P(e £ dt\e > 0) = Xe'^^dt. Let = to < h < ■ ■ ■ be 
a fixed infinite sequence of numbers, describing the times at which X is observed. Let 



Lt{u,xo,xi, . . 



1 



t exp 



[xi - - a{ti - V u)+f 



2{ti - ti_i) 



Then 



P{Xti G dxi for alH > 1 s.t. ti <t) = Lt{Q, xq, xi, 



Therefore, the conditional likelihood of the observations A^y, A^^, 
Lt{u) = Lt{u,Xt^,Xt^,. . .) 



I dxi. 
i>i,ti<t 

. , given 6 = n, is 



1 



t exp ■ 



[Xt, - Xt,_, - a{ti - ti.i V u)+f 
2(t/ - ti^i) 



To this point, we have already assumed that such a process X exists. The actual construction 
starts from a standard Brownian Motion A, and is then achieved via a change of measure. To that 
end, let (fi, J^, Pqo) support a standard Brownian Motion A and an independent random variable 
6 with Poo(0 = 0) = vr, and Poo(0 G dije > 0) = Xe'^^dt for t > 0. 

Then, Poo {Xti G dxi for alH > 1, < t} is 



Lt{oo,xo,xi,...) dxi= = 

l>iM<t i>i,tt<tV^^KU-ti-i) 



exp 



(x; - Xi-iY 
2{ti-ti.i) 



dx] 



for all t > 0. Let F be the filtration obtained by observing A at fixed times = to < < " " " ) and 
let G = {Gt)t>(} be the augmentation of F by o"(0), so that Qt = ^ (7(0). 
Define P on ^oo 5 locs-lly Sjlong the filtration, by 



dP 

dPZ. 




ti - ti- 



2{ti-ti-i) 



Under P, the random variables A^^ — Aj^ ^, I > 1, conditionally on 0, are independent and 
Gaussian with mean a {ti — (0 V tz-i))"^ and variance ti — t/_i. 
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Since Zq{Q) = 1, P and Pqo are identical on Gq = (t{Q), so that has the same distribution 
under P and Pqo- Under P, X has the distribution of a Brownian Motion which gains drift a at 
time O. We will work under P for the remainder of this Appendix. 

Define the conditional odds process 

* p{e>t\Tt) E^[Ztie)i{e>t}\:Ft]' 

with the second equality following from Bayes' Theorem. On the set {& > t}, {ti — (0 V = 
{ti - 9)+ = for ah ^ > < t. Therefore, Ztie)l{e>t} = l{0>t}- 
Thus, 

[Zt{Q)l{e>t}\J't] = Poo (e > t\Tt) 

= Poo{e>t) 



:i-TT)e 



-xt 



So, ( A.l ) becomes 
(A.2) 



Eoo [Zt{Q)l{e<t}\J^t 



[Zt{Q)\e<t^\Ft] . 



(1 — 7r)e~'^* 1 — vr 

We will now focus on this last term in ( A.2[ ). Write 

(A.3) ^oo [Zt{Q)l{Q<t}\:Ft] = 7rZt(0) + (1 - vr) / \e-^' Zt{u)du. 

Jo 

Suppose that t„_i < t < tn for some n > 1. By definition, Zt{u) = Zt^^_^{u) for every u > 0, and 
Zt„_i{u) = 1 for every tn-i < u < tn- This implies that (A.3) is 



^Zi_,(0) + (l-7r) [ Xe-^-Zt^_, 
Jo 



{u)du 



7rZi„_,(0) + (l-vr) 



Ae-^"Zt„_, {u)du + (1 - vr) / Ae^-'^Zt^, {u)du 



-\u I 



1 - vr 



-^t„.i + (l-vr)fe 



From this, it follows that, for tn-i ^ t < tn, we have that ( |A.2 ) is equal to 



and this establishes the first part of (2.1). We now derive the form of <I>t^, conditionally on ^tn-i- 
Since Zt„_^{u) = 1 for u > tn-i, we have 



Ztju) 



Zt^^^{u) exp ■ 



^n— 1 



((t„ - (wvi„_i))+y 

2 (^ri ^n— 1 ) 



,it > 0. 
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So, (A. 2) becomes 



l-TT 



(^TrZt„_, (0) + (1 - ^) ^ " ' Ae-^"Zt„_, {u)du^ exp | (Xt„ - J a - y (t„ - t„_i) 

in—1 '^(.^n ^n— l) 



+ (1 — TTj / Ae ^t„_i (u) exp < ^ — > du 



=1 



which is equal to 

«P { (X,., - a-^{t„- t„_i)| 



After making the substitution w = — (t„ — u) for the integral above, we see that this is equal to 



the second term in (2.1). 
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